MIDAS: Multi-LLM Iterative Data-Adaptive Summarization
Karen Lee, Dhanashree Balaram, Seojun Shon, Umair Rasheed
cs.CL, cs.AI, cs.MA
Submitted: 2026-08-05
Comments: Accepted at the 20th International Conference on Document Analysis and Recognition (ICDAR 2026). 17 pages, 2 figures
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Text summarization is deceptively difficult.
Terminology
Abstract
Text summarization is deceptively difficult. While condensing information seems straightforward, real-world enterprise summarization of support tickets, legal documents, incident reports, and more, demands strict adherence to domain-specific guidelines, output formats, and organizational conventions. Crafting prompts that reliably satisfy these constraints is labor-intensive, requiring significant human expertise and continuous maintenance as requirements evolve. Existing automated prompt optimization methods reduce this burden through Large Language Model (LLM) critique-driven refinement, yet remain limited by static prompts that cannot adapt to the diversity of summary applications. We propose Multi-LLM Iterative Data-Adaptive Summarization (MIDAS), a multi-LLM framework that extends this paradigm with data-driven pattern learning and use-case-specific personalization, enabling automatic adaptation to different summarization requirements without manual prompt engineering. Applied to enterprise customer ticket summarization across five output formats, MIDAS achieves the strongest overall performance against state-of-the-art critique-driven optimization frameworks such as CriSPO and ZERA, improving ROUGE-1 by up to 11.0%, ROUGE-2 by up to 18.2%, and ROUGE-L by up to 8.0%, while consistently improving BERTScore F1 across all formats and output types. We additionally demonstrate cross-model and cross-domain generalization through multi-LLM configurations and finance-domain summarization benchmarks.
Sources
- LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization
- A Comparative Study of Quality Evaluation Methods for Text Summarization
- Collaborative User Prompt for Personalized Generative Recommendation
- Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Models
- PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Models
- ProRefine: Inference-Time Prompt Refinement with Textual Feedback
- ZERA: Zero-init Instruction Evolving Refinement Agent -- From Zero Instructions to Structured Prompts via Principle-based Optimization
- BERTScore: Evaluating Text Generation with BERT
- A Survey of Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering