Peer review should constrain evaluative authority
cs.DL, cs.CR, cs.CY
Submitted: 2026-10-02
Updated: 2026-10-02
Comments: NeurIPS 2026 Position Paper Track
License: http://creativecommons.org/licenses/by/4.0/
The gist: Our position shifts the target of peer-review reform from reducing uncertainty through evaluator improvement or judgment aggregation to constraining evaluative authority under persistent uncertainty.
Abstract
Our position shifts the target of peer-review reform from reducing uncertainty through evaluator improvement or judgment aggregation to constraining evaluative authority under persistent uncertainty. Scientific peer review aims to assure research quality but also allocates scarce academic opportunities, such as publication in high-impact venues. However, the review system is under growing pressure as submissions increase while reviewer capacity, expertise, scrutiny, and incentives cannot scale accordingly. As peer review operates under persistent evaluative uncertainty, evaluator improvement and judgment aggregation remain structurally limited. We therefore propose authority-constrained peer review: before a consequential judgment is released as decision-relevant, it should withstand structured scrutiny through critique-authority decoupling, independent secondary evaluation, contestability and reversibility, and accountability for unsupported influence. Our aim is to prevent weakly supported judgments from entering final deliberation with the same authority as judgments that have withstood structured scrutiny, while creating incentives for reviewers to justify consequential claims more carefully. For large-scale computer-science conferences, this position offers an institutional response to uneven review quality and reviewer-assignment dependence; more broadly, it provides a governance principle for any domain where uncertain expert judgments allocate scarce academic opportunities.
Related papers
- Persistence Paradox in Dynamic Science: Evidence from the Deep Learning Revolution
- How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?
- MTRACE: Multilingual Retrieval-Augmented Generation for Temporally Diverse Text Corpora
- Eleven Years of BRACIS: A Meta-Scientific Study of the Brazilian Conference on Intelligent Systems
- Evidence-Based Scientific Question Discovery: A Framework with Historical Backtesting