LabourCrew: A Multi-Agent RAG Framework for Trustworthy Adversarial Deliberation and Statutory Reasoning over Labour Law
cs.CL, cs.AI, cs.IR
Submitted: 2026-08-18
Updated: 2026-08-18
License: http://creativecommons.org/licenses/by/4.0/
The gist: In statutory question answering, every claim must be traceable to evidence, not merely relevant, since unverifiable labour-rights answers carry serious legal consequences.
Terminology
Abstract
In statutory question answering, every claim must be traceable to evidence, not merely relevant, since unverifiable labour-rights answers carry serious legal consequences. Current systems fall short: single-pass RAG cannot detect insufficient evidence, while multi-agent legal-debate systems treat grounding as a prompting convention, letting agents cite unretrieved evidence. To address this gap, we introduce LabourCrew, a multi-agent RAG framework built around three grounding mechanisms: StatuteGraph, a graph index that explicitly links chapter, section, proviso, and cross-reference structure rather than fixed-length spans; an Evidence Exchange Protocol that confines advocates and an interpreter to an evidence ledger, making citation to unretrieved text impossible, while a fault-tolerant supervisor board runs advocates in parallel so individual failures degrade rather than crash the system; and a Calibrated Trust Gate that replaces categorical accept/reject decisions with a trust score, thresholded via conformal risk control for a distribution-free bound on the false-accept rate. We evaluate on LabourActQA, a 500-item Bangla question set from the Bangladesh Labour Act, 2006, spanning seven reasoning categories and three difficulty tiers. The framework drives the empirical false-accept rate to 0.081, within the target level (α= 0.10), achieves the highest Answer Relevancy among HyDE RAG, Graph-RAG, and Hierarchical RAG (0.862 > 0.839, 0.815, 0.828), and degrades gradually rather than catastrophically as question difficulty increases. These results show that calibrated abstention, not retrieval quality alone, is what makes legal question answering auditable in low-resource statutory domains.
Sources
- Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
- Conformal Risk Control
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
- LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- Retrieval-Augmented Generation with Graphs (GraphRAG)
- CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
- MASLegalBench: Benchmarking Multi-Agent Systems in Deductive Legal Reasoning
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
- Language Models with Conformal Factuality Guarantees
- Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
- ReAct: Synergizing Reasoning and Acting in Language Models
- L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering