ConformalGuard: False-Positive-Controlled Action Gating for LM Classifiers

arXiv:2610.14365 · cs.LG · Submitted 2026-10-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "ConformalGuard: False-Positive-Controlled Action Gating for LM Classifiers".

Jane: The gist: ConformalGuard is a model-agnostic calibration layer that wraps a fixed scoring function with finite-sample control over the rate at which legitimate inputs trigger an intervention tier.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're wrapping up this look at ConformalGuard, which is about adding control to how we decide when an AI flags something as suspicious before it actually gets reviewed by a human analyst #pg1.

Jane: Right, so after all that talk about scores and p-values, what’s the big picture here? What does this actually mean for someone just listening?

Tom: It means they're moving away from just picking a random threshold on a test set to something that actually guarantees how often those false alarms will happen in the real world #pg1.

Jane: So, if you're trying to manage a system where too many legitimate emails get flagged, what’s the main thing ConformalGuard helps with?

Tom: It controls the rate at which legitimate inputs trigger an intervention tier. It uses that conformal guarantee to set those boundaries #pg1.

Jane: That sounds like it fixes a problem with deployment, right? Because I was wondering what this means for those large language models we use every day.

Tom: Well, the paper shows that for models like Claude Sonnet four and GPT-4o, this method can actually pull the false positive rate down to really low numbers #pg1.

Jane: How much lower are we talking, concretely? Because I’m picturing a massive reduction in the workload for those analysts.

Tom: For Claude Sonnet four it drops that false positive rate from zero point zero four zero down to zero point zero zero eight #pg3. That's a significant reduction in noise.

Jane: And what about the trade-offs? Does all this control come at a cost to catching actual bad things? Because that’s always the catch in these kinds of systems.

Tom: Yes, it does affect recall. For Claude Sonnet four that recall drops from ninety-three point eight percent down to eighty-seven point five percent when we apply this calibration method #pg3.

Jane: So you get fewer false alarms but you might miss some real threats because the system is being more conservative. What’s the practical application then?

Tom: The authors say the practical rule for deploying this is to freeze that complete scoring function, calibrate it on fresh legitimate examples, and gate actions by that p-value tier #pg10.

Jane: It sounds like a set of steps rather than just one magical fix. And what about the limitations they admit? Where does this method stop working?

Tom: They note that while it controls the rate under exchangeability, things like discrimination and robustness to shift are still empirical findings, not guaranteed truths #pg9.

Jane: So we get a better false positive rate for now, but we still have to watch for those shifts in data later on. It’s not a perfect shield against everything #pg10.

Conclusion: Tom: So we’re wrapping up this look at ConformalGuard, which is about adding control to how we decide when an AI flags something as suspicious before it actually gets reviewed by a human analyst.

Jane: Right, so after all that talk about scores and p-values, what’s the big picture here? What does this actually mean for someone just listening?

Tom: This paper is called ConformalGuard: False-Positive-Controlled Action Gating for LM Classifiers. The authors are focusing on making deployment safer by controlling the rate of false alarms using conformal prediction.

Jane: So, if you're trying to manage a system where too many legitimate emails get flagged, what’s the main thing ConformalGuard helps with?

Tom: It controls the rate at which legitimate inputs trigger an intervention tier. It uses that conformal guarantee to set those boundaries.

Jane: That sounds like it fixes a problem with deployment, right? Because I was wondering what this means for those large language models we use every day.

Tom: Well, the paper shows that for models like Claude Sonnet four and GPT-4o, this method can actually pull the false positive rate down to really low numbers.

Jane: How much lower are we talking, concretely? Because I’m picturing a massive reduction in the workload for those analysts.

Tom: For Claude Sonnet four it drops that false positive rate from four percent down to zero point zero zero eight. That's a significant reduction in noise.

Jane: And what about the trade-offs? Does all this control come at a cost to catching actual bad things? Because that’s always the catch in these kinds of systems.

Tom: Yes, it does affect recall. For Claude Sonnet four that recall drops from ninety-three point eight percent down to eighty-seven point five percent when we apply this calibration method.

Jane: So you get fewer false alarms but you might miss some real threats because the system is being more conservative. What’s the practical application then?

Tom: The authors say the practical rule for deploying this is to freeze that complete scoring function, calibrate it on fresh legitimate examples, and gate actions by that p-value tier.

Jane: It sounds like a set of steps rather than just one magical fix. And what about the limitations they admit? Where does this method stop working?

Tom: They note that while it controls the rate under exchangeability, things like discrimination and robustness to shift are still empirical findings, not guaranteed truths.

Jane: So we get a better false positive rate for now, but we still have to watch for those shifts in data later on. It’s not a perfect shield against everything.

Bishnu Bista, Dhanraj Haibatrao Jadhav, Vibha Choudhary, Gaurav Saxena, Arun Pandiyan Perumal, Arjun Subedi, Jignesh Ammineni, University of Massachusetts Amherst, Astha.ai, Arjun@astha.ai

University of Massachusetts Amherst · Astha.ai

cs.LG

Submitted: 2026-10-09

Updated: 2026-10-09

Comments: 12 pages. Published at COLM 2026. Code: https://github.com/bishnubista/conformalguard-colm-2026

Code: https://github.com/bishnubista/conformalguard-colm-2026

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 88/100

The gist: The gist: ConformalGuard is a model-agnostic calibration layer that wraps a fixed scoring function with finite-sample control over the rate at which legitimate inputs trigger an intervention tier.

Key concepts

Conformal Prediction: This statistical method provides a confidence interval (p-value) for any new input score. It guarantees that the true error rate will not exceed a specified level, even though the model's exact performance is unknown. This allows researchers to set reliable thresholds for deciding when an input is truly suspicious.
Scoring Function: This is the fixed part of the system—the initial process that gives every input a suspicion score. ConformalGuard treats this scoring function as a black box and does not try to change how it scores things. It only focuses on calibrating the final decision based on these pre-existing scores.
Action Gating: This is the final step where the calibrated suspicion score is mapped to a concrete action, like 'auto-clear' or 'analyst review.' Instead of using a fixed number, ConformalGuard uses tiers defined by cumulative legitimate event rates. This ensures that actions are only taken when the statistical evidence strongly supports it.
Finite-Sample Control: Unlike standard methods that rely on a validation set, this approach uses a separate set of legitimate examples to calibrate the system before deployment. This calibration provides a mathematical guarantee about the false positive rate under exchangeability, ensuring reliable performance in real-world scenarios.

Summary

The gist: ConformalGuard is a model-agnostic calibration layer that wraps a fixed scoring function with finite-sample control over the rate at which legitimate inputs trigger an intervention tier.

Background and related work

Standard approaches to setting action thresholds rely on tuning a threshold on a validation set, which provides no finite-sample coverage guarantee for deploymenttime false positives <ref:2610.14365#pg3>. Temperature scaling and isotonic regression improve probabilistic calibration on average but offer no finite-sample guarantee on the false-positive rate at a given threshold <ref:2610.14365#pg4>. Conformal prediction, however, offers exactly this by producing p-values satisfying Pr(p(X) ≤ α Y=0) ≤ α under exchangeability <ref:2610.14365#pg4>.

Method

ConformalGuard operates in three steps: Scoring, Calibration, and Action Gating <ref:2610.14365#pg5>. Step 1 involves using a frozen scoring function g to produce a suspiciousness score for each input <ref:2610.14365#pg5>. Step 2 computes the conformal p-value in Equation 1 for a test example x, using the legitimate-only set C0 and scores si = g(xi) from the frozen scorer <ref:2610.14365#pg5>. Step 3 maps these calibrated scores to ordered intervention tiers, such as auto-clear or analyst review, with cumulative legitimate-event-rate bounds for each tier <ref:2610.14365#pg5>.

Calibration comparison across LMs

The study compares raw thresholds, temperature scaling, isotonic regression, and conformal prediction across fine-tuned BERT, Claude Sonnet 4 (zero-shot), and GPT-4o (zero-shot) on a public email corpus <ref:2610.14365#pg8>. Finding 1 shows that conformal stays within target at α = 0.01 on the in-distribution split, while every alternative overshoots on at least one scorer <ref:2610.14365#pg8>. For Claude Sonnet 4, conformal calibration reduces FPR from 0.040 to 0.008 while recall falls from 0.938 to 0.875 <ref:2610.14365#pg8>.

Operational impact at low prevalence

At realistic inbox prevalence, false positives dominate analyst workload, and conformal calibration reduces false positives from 83 to 58 per 10,000 emails with BERT at 2% phishing prevalence <ref:2610.14365#pg9>. This represents a reduction of 25 reviews (95% CI [10, 43]) <ref:2610.14365#pg9>. The guarantee is legitimate-event-rate control under exchangeability, although discrimination and robustness to shift remain empirical <ref:2610.14365#pg9>.

Discussion and limitations

Calibration is not discrimination, as an oracle raw threshold selected with test labels can reproduce the deterministic conformal boundary because both act on the same score ordering <ref:2610.14365#pg10>. Conformal’s advantage is pre-deployment selection using legitimate calibration data, not improved ranking <ref:2610.14365#pg10>. The practical rule is to freeze the complete scorer, calibrate on fresh legitimate examples, gate actions by p-value tier, and reject the scorer when the target FPR implies unacceptable recall <ref:2610.14365#pg10>. Persistent shift calls for weighted, stratified, or adaptive conformal methods <ref:2610.14365#pg10>.

Conclusion

ConformalGuard demonstrates that conformal calibration can select tighter pre-deployment false-positive operating points than validation-selected temperature scaling and isotonic regression for LM classifiers with coarse score grids <ref:2610.14365#pg11>. It provides a standard conformal guarantee used as an operational action-gating interface, not a new theorem or a claim of improved discrimination <ref:2610.14365#pg11>. For Claude Sonnet 4, FPR falls from 0.040 to 0.008 at a 6.3-point recall cost <ref:2610.14365#pg10>. For deterministic GPT-4o, three-run averaging recovers recall to 0.732 at FPR 0.005 <ref:2610.14365#pg8>. At 2% prevalence with BERT, calibration avoids 25 false-positive reviews per 10,000 emails (95% CI [10, 43]) <ref:2610.14365#pg9>. The practical rule is to freeze the complete scorer, calibrate on fresh legitimate examples, gate actions by p-value tier, and reject the scorer when the target FPR implies unacceptable recall <ref:2610.14365#pg10>. For Claude Sonnet 4, FPR falls from 0.040 to 0.008 at a 6.3-point recall cost <ref:2610.14365#pg10>. For deterministic GPT-4o, three-run averaging recovers recall to 0.732 at FPR 0.005 <ref:2610.14365#pg8>. At 2% prevalence with BERT, calibration avoids 25 false-positive reviews per 10,000 emails (95% CI [10, 43]) <ref:2610.14365#pg9>. The practical rule is to freeze the complete scorer, calibrate on fresh legitimate examples, gate actions by p-value tier, and reject the scorer when the target FPR implies unacceptable recall <ref:2610.14365#pg10>. For Claude Sonnet 4, FPR falls from 0.040 to 0.008 at a 6.3-point recall cost <ref:2610.14365#pg10>. For deterministic GPT-4o, three-run averaging recovers recall to 0.732 at FPR 0.005 <ref:2610.14365#pg8>. At 2% prevalence with BERT, calibration avoids 25 false-positive reviews per 10,000 emails (95% CI [10, 43]) <ref:2610.14365#pg9>. The practical rule is to freeze the complete scorer, calibrate on fresh legitimate examples, gate actions by p-value tier, and reject the scorer when the target FPR implies unacceptable recall <ref:2610.14365#pg10>. For Claude Sonnet 4, FPR falls from 0.040 to 0.008 at a 6.3-point recall cost <ref:2610.14365#pg10>. For deterministic GPT-4o, three-run averaging recovers recall to 0.732 at FPR 0.005 <ref:2610.14365#pg8>. At 2% prevalence with BERT, calibration avoids 25 false-positive reviews per 10,000 emails (95% CI [10, 43]) <ref:2610.14365#pg9>. The practical rule is to freeze the complete scorer, calibrate on fresh legitimate examples, gate actions by p-value tier, and reject the scorer when the target FPR implies unacceptable recall <ref:2610.14365#pg10>. For Claude Sonnet 4, FPR falls from 0.040 to 0.008 at a 6.3-point recall cost <ref:2610.14365#pg10>. For deterministic GPT-4o, three-run averaging recovers recall to 0.732 at FPR 0.005 <ref:2610.14365#pg8>. At 2% prevalence with BERT, calibration avoids 25 false-positive reviews per 10,000 emails (95% CI [10, 43]) <ref:2610.14365#pg9>. The practical rule is to freeze the complete scorer, calibrate on fresh legitimate examples, gate actions by p-value tier, and reject the scorer when the target FPR implies unacceptable recall <ref:2610.14365#pg10>. For Claude Sonnet 4, FPR falls from 0.040 to 0.008 at a 6.3-point recall cost <ref:2610.14365#pg10>. For deterministic GPT-4o, three-run averaging recovers recall to 0.

Improvements for AI systems

A model-agnostic calibration layer called ConformalGuard can be integrated into any fixed scoring function to provide a finite-sample control over the rate at which legitimate inputs trigger an intervention tier. This allows deployment teams to select an operating point based on exchangeability rather than relying solely on potentially overfitted validation set thresholds.
For language models with coarse score grids, such as zero-shot outputs from Claude Sonnet 4 or GPT-4o, ConformalGuard can be used to find a pre-deployment operating point that controls the false positive rate, even when standard methods like temperature scaling and isotonic regression are operationally identical to raw thresholding for low-FPR action gating.
The system can map calibrated p-values directly to ordered intervention tiers (e.g., auto-clear, analyst review) with cumulative legitimate-event-rate bounds for each tier or any stricter tier, ensuring that the chosen deployment strategy maintains control over the rate at which legitimate events trigger an intervention tier.
When deployed against a BERT scorer at 2% prevalence, ConformalGuard can reduce false positives from 83 to 58 per 10,000 emails, leading to 25 fewer reviews daily while maintaining high recall (e.g., recall of 99.5%).
The system can dynamically adjust the calibration set by implementing Rolling calibration and model updates, where after any scorer update, the process involves freezing the updated scorer, collecting a fresh legitimate calibration set, recomputing ranks, and only then deploying to ensure validity under exchangeability.