A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging
The study found that an optimizer's performance comparison between solvers is meaningless if its verification probes reveal the target identity. An exact Minimum Hitting Set optimizer and a greedy method yielded identical results because the verifier disclosed the answer before optimization could resolve ambiguity. The solution is a two-stage diagnosability gate requiring evidence eligibility and non-revelation before optimizing component selection.