Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length
cs.AI, cs.CL
Submitted: 2026-03-23
Updated: 2026-11-11
Code: https://github.com/jingxuanchen916/multi-instance-processing
Terminology
Sources
- Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities
- Large Language Model-based Data Science Agent: A Survey
- LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
- Why Do Large Language Models (LLMs) Struggle to Count Letters?
- Too Long, Didn't Model: Decomposing LLM Long-Context Understanding With Novels
- Towards Accurate and Efficient Document Analytics with Large Language Models
- A Comprehensive Survey on Long Context Language Modeling
- NeedleChain: Measuring Intact Context Comprehension Capability of Large Language Models
- DS-STAR: Data Science Agent for Solving Diverse Tasks across Heterogeneous Formats and Open-Ended Queries
- LLM-Based Data Science Agents: A Survey of Capabilities, Challenges, and Future Directions
- DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing
- The WiLI benchmark dataset for written language identification
- Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries
- LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation
- $\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
- A Survey of Large Language Models
- GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents