Correct Code, Broken Contributions? SWE-CC: Benchmarking Repository Policy Compliance for Coding Agents
cs.SE, cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: 39 pages, 6 figures. Benchmark and source code: https://github.com/dangtruong01/swe-cc-arxiv
Code: https://github.com/dangtruong01/swe-cc-arxiv
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- SWE-Bench+: Enhanced Coding Benchmark for LLMs
- SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents
- AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation
- The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering
- SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
- SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?
- Training Software Engineering Agents and Verifiers with SWE-Gym
- SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents
- Kimi K2.5: Visual Agentic Intelligence
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements
- A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities
- Does Pass Rate Tell the Whole Story? Evaluating Design Constraint Compliance in LLM-based Issue Resolution
- FeatureBench: Benchmarking Agentic Coding for Complex Feature Development
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties