Adaptive Query Routing: A Tier-Based Framework for Hybrid Retrieval Across Financial, Legal, and Medical Documents
cs.IR, cs.AI
Submitted: 2026-04-14
Updated: 2026-10-28
Comments: 12 pages, 4 figures, 6 tables. v2: method description corrected to match the experiment code; section-level retrieval metrics withdrawn because the original scorer ignored document identity; FinanceBench corpus clarified (supplied evidence snippets, first 50 public questions); prompt and output-length differences between configurations disclosed
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Retrieval-Augmented Generation (RAG) has become the standard paradigm for grounding Large Language Model outputs in external knowledge.
Terminology
Abstract
Retrieval-Augmented Generation (RAG) has become the standard paradigm for grounding Large Language Model outputs in external knowledge. Lumer et al. presented the first systematic evaluation comparing vector-based agentic RAG against hierarchical node-based reasoning systems for financial document QA across 1,200 SEC filings, finding vector-based systems achieved a 68% win rate. Concurrently, the PageIndex framework reported 98.7% accuracy on FinanceBench through purely reasoning-based retrieval. This paper extends their work by: (i) implementing and evaluating three retrieval configurations: Vector RAG, Tree Reasoning, and the proposed Adaptive Hybrid Retrieval (AHR), on a small controlled corpus of financial, legal, and medical documents; (ii) introducing a four-tier query complexity benchmark; and (iii) employing LLM-as-judge evaluation with GPT-4o-mini. Experiments reveal that Tree Reasoning achieves the highest overall score (0.900), but no single paradigm dominates across all tiers: AHR scores highest on cross-reference queries (0.850), all three methods tie on multi-section queries (0.929), and Vector RAG scores highest on the single multi-document query (0.900). Vector RAG has the lowest mean latency (1.61 s against 3.40 s for Tree Reasoning). On the first 50 publicly released FinanceBench questions, using the evidence snippets supplied with the benchmark as the retrieval corpus, Tree Reasoning scores 0.938, Hybrid AHR 0.901, and Vector RAG 0.821. Because the configurations also differ in document access, generation prompts and output length limits, these differences cannot be attributed to retrieval architecture alone. In these preliminary results the adaptive router does not outperform fixed tree reasoning overall (0.845 vs. 0.900); they motivate a larger evaluation of when query-based routing helps, rather than establishing its benefit.
Sources
- Rethinking Retrieval: From Traditional Retrieval Augmented Generation to Agentic and Non-Vector Reasoning Systems in the Financial Domain for Large Language Models
- Retrieval-Augmented Generation for Large Language Models: A Survey
- Passage Re-ranking with BERT
- A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
- Query Rewriting for Retrieval-Augmented Large Language Models
- Corrective Retrieval Augmented Generation
- RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing
- Retrieval-Augmented Generation for Natural Language Processing: A Survey
- Retrieval-Augmented Generation for AI-Generated Content: A Survey
- Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
- Evaluation of Retrieval-Augmented Generation: A Survey
- Improving Retrieval for RAG based Question Answering Models on Financial Documents
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- FinSage: A Multi-aspect RAG System for Financial Filings Question Answering
- Mitigating Hallucination in Financial Retrieval-Augmented Generation via Fine-Grained Knowledge Verification
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering
- Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-off
- GPT-4 Technical Report
- A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
Related papers
- UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction
- Revisiting Text Ranking in Deep Research
- GateSID: One Gate to Balance Semantic and Collaborative Signals across the Item Lifecycle
- SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce
- GINGER: Spicing Up Graph-Based Vector Search
- Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA