Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelity
cs.AI
Submitted: 2026-09-25
Updated: 2026-09-25
Comments: 14 pages, 13 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Despite the growing capabilities of large language models (LLMs), prompt design remains largely heuristic and ad hoc.
Terminology
Abstract
Despite the growing capabilities of large language models (LLMs), prompt design remains largely heuristic and ad hoc. This project will explore prompt minimization, the process of reducing prompts to their smallest, most information-dense form while preserving output fidelity. Practically, shorter prompts reduce computational overhead and inference latency, especially when large contexts, such as entire documents or codebases, are included unnecessarily. Further, longer prompts can damage LLM reasoning and accuracy. Theoretically, the existence of multiple prompts yielding equivalent outputs suggests a high degree of redundancy in the input space, raising fundamental questions about what information is essential to elicit specific model behaviors. We propose three variant frameworks to identify and evaluate minimal prompts and demonstrate that minimal prompts often produce outputs comparable to those of their longer counterparts. These findings suggest new directions for efficient prompt engineering and deepen our understanding of input compression in LLMs.
Sources
- Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
- Language Models are Few-Shot Learners
- Efficient Prompting Methods for Large Language Models: A Survey
- Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
- Learning to Compress Prompt in Natural Language Formats
- Towards A Rigorous Science of Interpretable Machine Learning
- Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
- LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
- Enhancing Robustness in Large Language Models: Prompting for Mitigating the Impact of Irrelevant Information
- The Power of Scale for Parameter-Efficient Prompt Tuning
- Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
- Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles
- A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
- The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
- Large Language Models Can Be Easily Distracted by Irrelevant Context
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
- Attention Is All You Need
- Resilience of Large Language Models for Noisy Instructions
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems