REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering".

Jane: Here's the gist of the paper. The gist: REAP proposes Recursive Evaluation and Adaptive Planning to enhance Retrieval-Augmented Generation for multi-hop question answering by maintaining structured sub-tasks and facts through a recursive feedback loop between a Sub-task Planner (SP) and a Fact Extractor (FE).

Tom: REAP Framework Overview REAP reframes the multi-hop reasoning process from a linear pipeline into a dynamic, state-driven loop, designed around the core principle of explicitly decoupling planning from fact-gathering to enhance robustness and interpretability It primarily consists of two synergistic modules operating in a recursive loop: the Sub-task Planner (SP) and the Fact Extractor (FE) The overall workflow proceeds as follows: Given a complex query Q, an initial Decomposer first generates a structured task plan, P0 =

p1, p2,..., pN: Each sub-task pi ∈ P0 is a tuple (idi, qi, depsi), representing its unique identifier, query, and dependencies This plan, along with an initially empty facts list F0 = ∅, defines the initial state of the framework The core of REAP is an iterative process that continues until a termination condition is met At each step t, the SP analyzes the current state (Pt−1, Ft−1) to determine a set of executable next actions, Actionst For each sub-task pi ∈ Actionst, the FE performs retrieval and grounded reasoning to produce a new fact object, fi The facts list is updated: Ft = Ft−1 ∪

f1, f2,...: The SP then receives this feedback to generate the subsequent plan, Pt This recursive loop, where the SP guides the FE and the FE’s findings inform the SP’s next planning cycle, continues until the plan is fully resolved Finally, a Synthesizer generates the conclusive answer A by reasoning over the final, comprehensive facts list Ff inal and the original query Q.

Jane: Sub-task Planner (SP) The SP serves as the strategic core of the REAP framework, designed to overcome the limitations of myopic, step-by-step reasoning that often leads to local impasses or deadlocks By receiving and dynamically maintaining the complete initial task plan P0, the SP retains a global perspective throughout the process A key innovation of the SP is its state-aware, modular design Based on the fulfillment level lt of the new fact ft returned by the FE, the SP dispatches the task to one of two specialized sub-modules 1.

Tom: Plan Updater: This sub-module handles ideal scenarios where reasoning progresses as expected (e.g., lt is DirectAnswer) It performs deterministic and rulebased updates to maintain the task plan Its primary functions are Fact Substitution, which systematically rewrites pending sub-tasks by substituting abstract placeholders with concrete entities derived from newly acquired facts and Plan Forking, which in cases where a sub-task yields a set of multiple valid sub-answers, programmatically duplicates the subsequent dependent sub-tasks into parallel branches 2.

Paper summary: Tom: So, we’re looking at this paper called "REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering". Essentially, they’re tackling the problem that current retrieval augmented generation methods struggle with when you have complex multi-hop questions. The main issue is that these existing methods often lack a global plan, which means they can get stuck in local reasoning impasses.

Jane: That sounds like a big headache for any AI trying to answer tricky questions, right? So what REAP claims is that by explicitly maintaining structured sub-tasks and facts through two modules—a Sub-task Planner and a Fact Extractor—they can overcome those limitations. It’s about giving the system a global perspective to guide the whole reasoning direction.

Lu: What I find interesting is how they propose this recursive loop between these two modules, where the planner guides the extraction, and what REAP achieves is that it dynamically optimizes the task-solving trajectory as it goes. They focus on ensuring that both information retrieval and reasoning are tightly coordinated to build a reliable global context.

Meng: From a deployment standpoint, I’m watching how they handle that coordination. The idea of decoupling planning from fact-gathering sounds robust, but we need to see how stable this structure is when the retrieval returns ambiguous or low-quality documents at scale. What breaks at scale is usually in those recovery mechanisms.

Lalam: From my perspective, the paper focuses on creating a representation of global knowledge that gets incrementally enriched through these steps. The goal isn't just getting one answer, but building a logically coherent knowledge base that improves with every fact extracted.

Tom: Exactly, Lalam. And they introduce this unified task paradigm design which they fine-tune across different tasks to make the planning modules much more effective on complex, data-scarce questions where you don't have tons of specific training data for the reasoning steps themselves.

Jane: So it’s not just a single fix for one problem, but a framework designed to be adaptable. It seems like they are trying to make the system smarter about when to plan differently and when to recover from a bad path in the reasoning process.

Lu: The structure itself is what catches my eye here, especially how the Sub-task Planner handles different fulfillment levels of facts returned by the Fact Extractor. It’s not just one way to proceed; it branches into plan updating or re-planning depending on what the fact extraction actually yields.

Meng: Branching sounds complicated to monitor in production. If we have too many parallel branches, tracking which path is most cost-effective becomes a major operational challenge for you, Tom.

Lalam: But the structure is designed to be interpretable by maintaining that structured sub-task plan throughout the entire process. It makes tracing where a specific fact came from much clearer than just looking at one monolithic output.

Tom: Right, so they’re trading a simple linear pipeline for this dynamic, state-driven loop to improve reliability in these tricky multi-hop scenarios. This is definitely aimed at making RAG more dependable for harder tasks.

Conclusion: Jane: So, wrapping up this look at "REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering," the authors are Zhu, Zhou, Hong, Liu, and Wang. It’s a framework that essentially rethinks how we combine retrieval with reasoning for multi-hop tasks.

Tom: What this means in plain terms is that instead of just throwing the retrieved text into one big prompt and hoping for the best, you get a system that actively plans its next move based on what information it has gathered so far. It's about making the AI think recursively about which piece of evidence it needs next to solve the whole question.

Lu: The core contribution here is that they introduce this dual-module system—the Sub-task Planner and the Fact Extractor—to ensure that the reasoning trajectory is guided globally, while simultaneously expanding the factual context with reliable pieces of information at each step.

Meng: For a developer or someone shipping this to production, what I see is an attempt to make the planning component more robust against failures in the retrieval step. If one path fails, there's a designated module that can prune that branch and inject a new, logical sequence of steps instead of just crashing. That’s where the operational value lies for me.

Lalam: The implication for us is that we can expect models to handle more complex dependencies in questions because they aren't relying on a single, potentially flawed chain of thought from start to finish. They are building a more traceable and reliable knowledge base throughout the process.

Jane: It suggests that for multi-hop reasoning, the architecture needs to evolve beyond simple retrieval; it needs active, adaptive planning built into the generation loop itself to handle uncertainty better.