RH-RAG: Trustworthy Long-Form Generation for Privacy-Constrained Settings
Raj Shekhar Singh
cs.CL
Submitted: 2026-08-02
Comments: accepted in KDD 2026 SeT-LLM Workshop
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generating long-form content from extensive internal reports remains challenging for organizations operating under strict privacy and security constraints, where proprietary cloud-based LLM APIs are
Terminology
Abstract
Generating long-form content from extensive internal reports remains challenging for organizations operating under strict privacy and security constraints, where proprietary cloud-based LLM APIs are often not viable. While locally deployed open-weight models offer a privacy-preserving alternative, existing retrieval-augmented generation (RAG) approaches on smaller models frequently lack effective global planning and accumulate factual inconsistencies over long outputs. To address these limitations, we present RH-RAG, a multi-agent framework for secure and trustworthy long form generation using local language models. RH-RAG decomposes generation into three coordinated stages: a Planner Agent that constructs a global document outline from high-level semantic summaries, a Writer Agent that incrementally generates coherent section-wise content using bounded coherence memory, and a Checker Agent that mitigates hallucinations through natural language inference-based factual verification and an attestation-driven revision loop. The framework further employs a dual-level retrieval index that supports efficient planning and fine-grained contextual generation on consumer-grade hardware. Evaluations across literary, financial, and legal domains demonstrate that RH-RAG consistently improves factual grounding, semantic coherence, and document-level alignment compared to standard and hierarchical RAG baselines, while achieving reliability competitive with proprietary cloud-based systems without compromising data privacy.
Sources
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Learning to Plan and Generate Text with Citations
- REPLUG: Retrieval-Augmented Black-Box Language Models
- Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
- MIRAGE: Scaling Test-Time Inference with Parallel Graph-Retrieval-Augmented Reasoning Chains
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
- SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization
- Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities
- Mistral 7B
- The Llama 3 Herd of Models
- Qwen2.5 Technical Report
- Leveraging Large Language Models in Code Question Answering: Baselines and Issues
- Reliable, Adaptable, and Attributable Language Models with Retrieval
- Longformer: The Long-Document Transformer
- Unlimiformer: Long-Range Transformers with Unlimited Length Input
- GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse state
- Extending Context Window of Large Language Models via Positional Interpolation
- LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
- Generating Long Sequences with Sparse Transformers
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering