Longtail-SWE: High-Quality SWE Tasks for Underrepresented Languages
cs.SE, cs.AI
Submitted: 2026-11-06
Updated: 2026-11-11
Code: https://github.com/SWE-rebench/Longtail-SWE
Project page: https://kotlinlang.org/benchmark
Terminology
Sources
- SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents
- SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale
- SWE-Universe: Scale Real-World Verifiable Environments to Millions
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?
- MobileDev-Bench: A Benchmark for Issue Resolution in Mobile Application Development
- SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks
- DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
- R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- Kimi K3: Open Frontier Intelligence
- SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
- SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation
- Auditing Reward Hackability in Code RL Training Environments
- SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents
- SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring
- SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering?
- The Verification Horizon: No Silver Bullet for Coding Agent Rewards
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters