ProReviewer: Evidence-Driven Investigation for Scientific Peer Review

arXiv:2606.13349 · cs.CL · Submitted 2026-06-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "ProReviewer: Evidence-Driven Investigation for Scientific Peer Review".

Jane: The gist: ProReviewer, an agent that investigates scientific papers proactively by maintaining a structured review log and formulating the process as a Markov Decision Process (MDP),

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, we’re looking at this paper called "ProReviewer: Evidence-Driven Investigation for Scientific Peer Review". The main idea here is that existing AI review systems are kind of passive; they just generate a review based on what they already know. This paper argues that instead, we need an agent that can actually investigate things proactively based on new evidence as it comes in.

Jane: Right, so the core claim seems to be that this agent can adapt its investigation depth depending on what it finds. It frames the entire review process as a Markov Decision Process, which is basically a way of thinking about making sequential decisions where each decision changes what you look at next.

Lu: Exactly. They propose this ProReviewer agent which maintains a structured review log that acts as its state in this MDP formulation. This log isn't just notes; it’s supposed to be the workspace where the agent tracks all the evidence, questions, and intermediate findings during the review process.

Meng: So, if I get it right, instead of following a fixed pipeline like 'read section A then section B', this AI agent can decide to jump to a totally different part of the paper if it finds something suspicious or contradictory in an earlier section. That sounds much more flexible than what we usually build.

Lalam: And the structured log itself is key, because it records three specific types of evidence: claims, questions, and notes. These entries allow the agent to track exactly what it thinks is true right now and how those beliefs are being updated by new information as it reads the manuscript.

Tom: The paper suggests that this structure lets the agent revise earlier findings or resolve open questions selectively without having to re-read everything from scratch every time a new piece of data appears. It’s about selective revision guided by the log, not just accumulating unstructured traces.

Jane: So what matters for us is that this isn't just another model generating text; it’s an agent making investigative decisions based on a history of findings. They say this approach allows the review strategy itself to be learned through reinforcement learning rather than being hand-designed by humans.

Lu: That reinforcement learning aspect, combined with the MDP, is what they use to train ProReviewer. They use supervised fine-tuning on some synthesized interaction trajectories before moving into a policy optimization phase where it learns how to navigate this review space.

Meng: From a deployment standpoint, if you’re running this on real traffic, the reward structure they have seems pretty detailed. It looks like they reward syntactic validity for formatting errors and review completeness, but also quality based on depth and grounding metrics.

Lalam: And there's also a score alignment component in the reward function. This means the agent isn't just trying to be thorough; it’s also learning to match its own assessment of the paper against what a human reviewer would generally give.

Tom: So, we’re moving from passive generation to active investigation guided by this structured log, and the training is designed to make that investigation adaptive and evidence-driven. This sets up a really interesting direction for how we approach complex document analysis.

Conclusion: Jane: Thinking back to the full paper on "ProReviewer: Evidence-Driven Investigation for Scientific Peer Review", the big picture here is shifting automated review from just summarizing what’s there to actually investigating it like a human expert would when they have doubts or find inconsistencies.

Tom: Yeah, and the authors are saying that by using this Markov Decision Process setup and that structured log, you get reviews that aren't just good at sounding right, but actually traceable back to the specific parts of the paper that support each critique.

Lu: It moves away from fixed workflows where you just follow a script through sections. Instead, the agent learns *how* to investigate by seeing what evidence leads to which conclusions and adjusting its path accordingly.

Meng: For us in production, this means if a model flags something as weak in one section, it can immediately go check the related claims in another section instead of just moving on to the next planned task. That adaptability is what makes it useful for complex documents where things aren't always linear.

Lalam: The implication for me is that we can build systems where every critique has a verifiable chain of support, which really helps when you need to justify why an AI flagged something as important. It’s about accountability in the review process.

Jane: So, in simple terms, it means the AI isn't just writing a review; it’s performing a targeted search for evidence to back up its claims while simultaneously figuring out what questions remain unanswered. This is how we move toward more robust AI assistance for scientific literature.

Tom: It really comes down to making the process proactive—the agent looks for things that look suspicious and then uses the structured log to decide exactly which part of the paper needs inspection next, based on what it’s already found. That evidence-driven approach seems like a solid way forward for these kinds of deep analytical tasks.

Haishuo Fang, Yue Feng, Iryna Gurevych

Ubiquitous Knowledge Processing Lab (UKP Lab) · National Research Center for Applied Cybersecurity ATHENE · School of Computer Science, University of Birmingham

cs.CL

Submitted: 2026-06-11

Updated: 2026-10-14

Code: https://github.com/UKPLab/arxiv2026-ProReviewer

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 63/100

The gist: The gist: ProReviewer, an agent that investigates scientific papers proactively by maintaining a structured review log and formulating the process as a Markov Decision Process (MDP), outperforms

Key concepts

Markov Decision Process (MDP): An MDP is a mathematical framework used by ProReviewer to model the review process. It defines the state (what the agent knows), actions (what it does, like reading a section), transitions between states, and rewards. This structure allows the agent to make sequential decisions about how to investigate a paper step-by-step.
Structured Review Log (L): This log is the central workspace where ProReviewer stores all accumulated evidence from reading the paper. It categorizes findings into Claims (assertions with verification status), Questions (unresolved queries), and Notes (intermediate thoughts). This structured record ensures that every final review point can be traced back to specific, verifiable evidence.
Evidence-Driven Investigation: This approach means the agent's next investigative step is not fixed but depends entirely on the evidence it has already gathered. Instead of following a rigid workflow, ProReviewer uses accumulated findings—like resolving a claim or finding corroboration—to decide what to inspect next, leading to adaptive and deep analysis.
Reward Structure: The reward system guides the agent's learning by assigning points at different levels. Step-level rewards check for syntactic correctness and completeness, while trajectory-level rewards evaluate the final review quality based on depth, grounding accuracy, and alignment with human reviewer scores.

Terminology

Fine-Tuning — 1× in this paper · explained in 5 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes
Markov Decision Process (MDP) — 1× in this paper · explained in 15 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes

Summary

The gist: ProReviewer, an agent that investigates scientific papers proactively by maintaining a structured review log and formulating the process as a Markov Decision Process (MDP), outperforms existing methods by enabling evidence-grounded, traceable critiques.

How it works

The core mechanism is formulated as a Markov Decision Process (MDP) denoted as M = (S, A, T, E, R) where the state S captures the agent’s understanding at step t: (1) the current context Ct, containing the most recent action and observation (e.g., a section); (2) the review log Lt that records the agent’s accumulated evidence entries (§3.2); and (3) the paper index P, comprising the title and table of contents, which guides navigation through the paper>.
The action space A reflects two complementary activities: environment actions acquire information: read section retrieves the full text of a section, look up searches the paper for specific keywords, and finish terminates the episode; log actions maintain the review log L (§3.2): log records new evidence entries (claims, questions, or notes), update revises the status of existing entries as new information emerges, and outline constructs the final review by adding points that cite accumulated evidence>.
The structured review log L is a workspace that records three types of evidence entries: Claims (C1, C2,...): assertions from the paper, annotated with source section and a verification status (e.g., supported, weak, invalid); Questions (Q1, Q2,...): questions raised during reading, each with a resolution status (e.g., open); and Notes (N1, N2,...): free-form intermediate findings and thoughts>.
The agent builds L incrementally via the log action and refines earlier entries via update as new evidence emerges (e.g., marking a claim as supported after finding corroboration in a later section) to produce the final review, where each point is tagged with the IDs of the evidence entries that support it, creating a verifiable chain from critiques back to specific paper content>.

Reward Structure

The total reward rt at each step t is defined as rt = wsyn · rsyn t + Pk∈K wk · rk, where trajectorylevel rewards are computed at episode termination and broadcast uniformly to all steps>. The step-level reward rsyn t includes syntactic validity (r synt), which penalizes formatting errors, execution errors, and grounding errors; review completeness (r fmt), which checks for the presence of a summary, strength, weakness, and overall score; review content quality (r qual = α · r depth + (1 − α) · r grd) where depth evaluates engagement with methodological details and grounding measures whether critiques are grounded in concrete content rather than hallucination; and score alignment (r scr), which encourages the agent to align its score sˆ with the human reviewer average s¯>.

Training Procedure

ProReviewer is trained using a two-stage pipeline: first, supervised fine-tuning (SFT) on interaction trajectories distilled from a strong teacher model (Qwen3.5-397B-A17B); second, Group Relative Policy Optimization (GRPO) reinforcement learning with a multi-dimensional reward (§3.3). The training follows a two-phase curriculum: Phase 1 trains with only deterministic, rule-based rewards (syntactic validity, review completeness, and score alignment); Phase 2 retains all Phase 1 rewards with adjusted score alignment weight and additionally introduces the LLM-judge-based content quality reward (technical depth and grounding) >.

Evaluation Metrics

Performance is assessed across four quality dimensions using LLM-as-a-judge evaluation, which aggregates scores from three diverse judges: GPT-5.4 nano, DeepSeekV4 flash, and RevUtil >. The dimensions are Actionability (assessing if suggestions are explicit or concrete), Grounding (measuring reference to specific parts of the paper), Verifiability (assessing logical justification), and Technical Depth (evaluating engagement with methodology) >. Furthermore, a counterfactual error detection task is used to test the ability to detect subtle logic errors deliberately embedded in manuscripts, where success is defined by identifying an injected error as a weakness in the generated review>.

Key Findings

Experiments show that ProReviewer with an 8B backbone ranks first on average across five review quality dimensions, improving over frontier LLM-based systems by up to 39% relatively and over the best fine-tuned baseline by 16% >. It is also confirmed that ProReviewer more effectively detects subtle cross-section inconsistencies (§5.2) and maintains robust performance as paper length increases (§5.3) >. The structured review log is essential, as ablating it causes performance drops across all quality dimensions, suggesting that structured tracking of claims, questions, notes is essential for ProReviewer to produce well-grounded and technically substantive reviews.

Conclusion

ProReviewer shifts automated peer review from passive generation to proactive investigation by formulating the review process as an MDP guided by a structured review log >. This approach allows the agent to adapt its investigation depth based on accumulated evidence, leading to reviews that are traceable, evidence-grounded, and superior across automatic and human evaluations >. The results suggest that proactive investigation supported by evidence tracking is a promising direction for LLM-assisted peer review and potentially for tasks requiring multi-step analytical reasoning over complex documents>.

--- Page 1 ---

The structured review log L is a workspace that records three types of evidence entries: Claims (C1, C2,...): assertions from the paper, annotated with source section and a verification status (e.g., supported, weak, invalid); Questions (Q1, Q2,...) questions raised during reading, each with a resolution status (e.g., open); and Notes (N1, N2,...) free-form intermediate findings and thoughts>.

Improvements for AI systems

Proactive Investigation via MDP Formulation: Implement a Markov Decision Process (MDP) framework to allow agents to decide at each step which section to read, what evidence to extract, and when to revisit earlier content through a learnable policy. This enables the agent to move beyond fixed investigation path by adapting its strategy based on accumulated evidence.
Structured Evidence Tracking: Maintain a structured review log that records Claims (C1, C2,...), Questions (Q1, Q2,...), and Notes (N1, N2, N3), ensuring each critique is traceable to its supporting evidence. This prevents hallucinations by requiring every review point to cite corresponding evidence IDs.
Multi-dimensional Reward Optimization: Optimize the agent using a multi-dimensional reward function that covers action validity at each step and overall quality metrics like Technical depth and Grounding, ensuring the system learns not just to generate text, but to produce reviews with substantive engagement with technical content.
Cross-sectional Inconsistency Detection: Enhance the agent's capability to detect logical errors by training it on counterfactual datasets, allowing it to perform in-depth reviewing through a process where any weakness identified by a method correctly matches the injected error.
Robustness to Paper Length: Design the system with a mechanism that maintains performance across varying paper sizes, as shown by ProReviewer's ability to maintain its lead across every length bin, demonstrating that the structured log is essential for handling longer manuscripts.

Abstract

Effective peer review requires adaptive investigation that responds to evidence emerging from each paper. Existing LLM-based reviewers largely organize the review process around predefined workflows, leaving limited scope for emerging evidence to reshape investigation. We propose ProReviewer, an evidence-driven review agent that formulates reviewing as a Markov Decision Process, with a structured review log forming the core of its state. At each step, the agent uses accumulated findings to decide what to inspect next and when to conclude. As new evidence emerges, it updates the log by revising claim statuses, resolving open questions, and recording new findings. Each point in the final review cites supporting log entries, making its judgments traceable to the recorded evidence. On 1,000 held-out ICLR 2026 papers, the trained 8B ProReviewer achieves the highest average score among all compared methods, including those built on a 397B model; human evaluators prefer its reviews over all baselines on three of four review-quality dimensions.