Programmable Distributions of Semantics-Preserving Variants for Evaluating Code Agents
cs.AI, cs.SE
Submitted: 2026-08-18
Updated: 2026-11-12
Code: https://github.com/CSU-TrustLab/jagged-frontier
Terminology
Sources
- Lumos: Let there be Language Model System Certification
- Certifying Knowledge Comprehension in LLMs
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
- Kimi K2.5: Visual Agentic Intelligence
- Bypassing the Safety Training of Open-Source LLMs with Priming Attacks
- Quantitative Certification of Agentic Tool Selection
- PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents