The Best Second Opinion Is Right Where the Primary Is Wrong: Partner Selection for Selective Span Tagging

arXiv:2610.37678 · cs.CL, cs.LG · Submitted 2026-10-12 · Read on arXiv

cs.CL, cs.LG

Submitted: 2026-10-12

Updated: 2026-10-12

License: http://creativecommons.org/licenses/by/4.0/

The gist: A span tagger that ranks its outputs by risk can defer the riskiest ones to a person, and a second model can sharpen this ranking.

Abstract

A span tagger that ranks its outputs by risk can defer the riskiest ones to a person, and a second model can sharpen this ranking. Which second model should it be? Partner selection is conditional, not global: the best second opinion is the model that is right where the primary is wrong. Across eight tagger types, three seeds and three English benchmarks (576 evaluated primary-partner pairings), choosing the partner by held-out accuracy on the primary's errors recovers 82% of the best partner's gain, against 69-70% for span F1 or a random pick. Frozen before a fourth dataset was chosen, the rule replicated there (Spearman 0.40 vs. 0.04 for span F1). Partner value is also directional. The predictive distribution is the second opinion: hard labels discard half of a partner's value, and scoring the served label improves 94% of pairings. Fixed before the fourth dataset was chosen, this score also replicated there. For LoRA-tuned decoders, independently trained partners beat every readout of the primary network. Applied zero-shot to 51 MASSIVE locales, every primary has a helpful trained partner in every locale, and the preferred partner follows competence.

Related papers