Query-Guided RoPE Modulation for Lost-in-the-Middle Mitigation in Long-Context LLMs
cs.CL, cs.AI, cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/iZHpy/QGPM
Terminology
Sources
- GPT-4 Technical Report
- Extending Context Window of Large Language Models via Positional Interpolation
- LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
- The Llama 3 Herd of Models
- Instruction Following by Principled Boosting Attention of Large Language Models
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
- NoLiMa: Long-Context Evaluation Beyond Literal Matching
- Attention Sorting Combats Recency Bias In Long Context Language Models
- Qwen3 Technical Report
- Qwen2.5 Technical Report
- Qwen2.5-1M Technical Report
- Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data