Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims

arXiv:2605.10575 · cs.CR, cs.AI, cs.LG · Submitted 2026-05-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims".

Elias: Here's the gist of the paper. Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer <ref:2605.10575#pg2> The paper introduces Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims <ref:2605.10575#pg3> This protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass <ref:2605.10575#pg4> The resulting audit on Gemma-2-2B-it shows that SafeLoRA fails the full-card pass under strict mechanism coding and still fails three of four under a permissive shrinkage relabel <ref:2605.10575#pg5> The Acceptance Standard A minimum acceptance standard for this claim is defined as a fine-tuning-time recipe reducing an installed gap in a way that is statistically reliable, semantically fresh, mechanism-consistent, and transferable <ref:2605.10575#pg3> This standard requires four diagnostic gates to pass strictly for a full-card pass <ref:2605.10575#pg4> The four diagnostics are Diagnostic (a) catching finite-split and template-level noise, Diagnostic (b) catching subject- or prompt-split artifacts, Diagnostic (c) checking mechanism class consistency via the parameter-space signature ρAT, and Diagnostic (d) testing transfer to a second installable task <ref:2605.10575#pg4> The conjunction rule requires all four diagnostics to pass strictly for a full-card pass <ref:2605.10575#pg4> The Four Diagnostic Gates The paper defines the thresholds for each diagnostic gate, which are conservative and designed to block specific over-readings <ref:2605.10575#pg4> Diagnostic (a) uses Question-Clustered Bootstrap, requiring the upper 95% CI on ∆gap to be below zero <ref:2605.10575#pg4> Diagnostic (b) requires both the paired and independent signed checks to hold, demanding that ∆gap + 2SEpaired Diagnostic (c) uses the parameter-space signature ρAT, where shrinkage passes if ρAT ≥ 0.6 and attack-targeted passes if ρAT Diagnostic (d) tests cross-task transfer, requiring conditions like ΓP > 0 and ΓX − ΓP ≤ 2SEcombined to pass <ref:2605.10575#pg4> Audit Results and Verdicts The audit tests whether available defense evidence supports the full-card pass, not whether any recipe is deployable <ref:2605.10575#pg4> The audit results in a 46-cell matrix where no cell satisfies the strict conjunction for a full-card pass <ref:2605.10575#pg5> The closest family is AC-AdamW, which passes (a) and (c) but fails (b) on the fresh-subject threshold and lacks a strict (d) pass <ref:2605.10575#pg5> The paper reports four verdict labels: full-card pass, near miss, missing evidence, and undefined <ref:2605.10575#pg4> Mechanism Diagnostic The parameter-space signature ρAT is introduced as the new mechanism-class diagnostic among the four <ref:2605.10575#pg5> The pass rule for (c) states that a cell’s claimed class matches the signature in (3): shrinkage passes if ρAT ≥ 0.6, while attack-targeted passes if ρAT The diagnostic flags claimed attack-aware updates whose parameter-space signature aligns with AdamW shrinkage <ref:2605.10575#pg5> Limitations and Intended Use The audit has empirical limits, as it is primarily based on Gemma-2-2B and uses cross-architecture runs as spot-checks rather than a scaling proof <ref:2605.10575#pg4> The standard is meant to reduce overclaiming and make partial results reportable, scoping SafeLoRA’s verdict to this installed-gap audit on Gemma-2-2B-it rather than a global judgment <ref:2605.10575#pg5> The paper emphasizes that missing diagnostics block the full-card label but are not counted as empirical evidence of failure on that gate <ref:2605.10575#pg5> Safe fine-tuning defense claims should not rest on held-out gap reductions alone <ref:2605.10575#pg5> Acceptance Cards type evidence for one claim under four diagnostics, and bundle the protocol with a documentation object and an executable audit package <ref:2605.10575#pg5> Partial passes remain reportable evidence; missing diagnostics are missing evidence, not implicit success <ref:2605.10575#pg5> The closest family is a near miss with deployment-accuracy cost, and SafeLoRA does not satisfy the full-card pass under the installed-gap protocol <ref:2605.10575#pg5> The standard is strict but satisfiable, as each gate targets a separate failure mode <ref:2605.10575#pg5> The gap between a 15.6pp training-distribution reduction and a 5.2pp fresh-domain reduction is itself informative <ref:2605.10575#pg5> The standard is built to detect the generalization failure that closes the gap on trained topics without addressing the underlying routing mechanism <ref:2605.

Title and authors: Nadia: Let's talk about the title and who wrote this thing: "Acceptance Cards: A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims". It’s a pretty descriptive name, suggesting they are building a formal system of evaluation rather than just another benchmark.

Elias: I think that title tells you right away that this isn't just about measuring the final gap reduction on a held-out set; it’s about creating an entire protocol—documentation, an audit package, and a standard—for how we assess those claims.

Priya: From my side, I’m focused on what that means for data trust. If you have a defense claim, you need to know exactly what evidence is required to back it up before you treat it as valid data. This sounds like the kind of formal structure needed when dealing with complex AI artifacts.

Nadia: Right. They are defining this minimum acceptance standard as reducing the installed gap in a statistically reliable way that is also semantically fresh, mechanism consistent, and transferable across different tasks or models.

Elias: And they validate this standard on Gemma-two-2B-it, while using other models like Qwen2 point 5-1 point 5B-Instruct, Llama three 8B, and Phi three mini as spot checks to see if the validation holds up across different architectures <ref:2605.10575#pg2,Qwen2.5-1.5B-Instruct, Llama 3 8B, and Phi 3 mini>.

Priya: I wonder how those cross architecture runs play into this. If a defense works on one model but fails the transfer test on another, does that invalidate the whole claim? It seems they are testing for a kind of robustness that goes beyond just one model family.

Nadia: They are testing for mechanism alignment and cross-task transfer as part of those four gates, so they’re explicitly looking to see if the defense works generally or just in a very specific setup.

Elias: The contribution isn't a new defense itself, which is important because it keeps the focus on evaluation rather than just another claim to be defended. It’s an evaluation protocol and an evidential standard for a narrow claim, not a general safe fine-tuning benchmark.

Priya: So, if I’m building a system that claims it has this defense, I can use these Acceptance Cards as the template for how I document my evidence instead of just presenting the raw gap number in isolation.

Nadia: Exactly. It gives you a clear structure for what needs to be measured and what remains missing when you evaluate those defense claims in practice.

The paper's summary: Nadia: So, to summarize the main idea of "Acceptance Cards: A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims," they are proposing a concrete set of tests to validate if a claim about a fine-tuning defense is actually sound.

Elias: The summary boils down to this: before you treat a recipe that reduces the gap as proof of safety, you have to pass four diagnostics: statistical reliability, fresh semantic generalization, mechanism consistency, and cross-task transfer.

Priya: They are saying that just seeing a smaller installed gap isn't enough because that reduction could be due to noise or artifacts. The paper argues that you need to prove the reduction is real in multiple ways simultaneously before you give it credit.

Nadia: That’s right, and they define the installation gap as the difference between the deployment framed rate and the evaluation framed rate, which is then compared to a matched baseline of zero. They use that delta gap to test if a recipe has reduced that installed gap in a way that isn't just luck.

Elias: The four diagnostic gates are what enforce this: they require all four diagnostics to pass strictly for you to get the full card pass, which is the minimum acceptance standard they set.

Priya: I’m thinking about how those gates translate into real data checks. Diagnostic one handles things like template-level noise and finite split issues, while diagnostic two deals with subject or prompt split artifacts that can skew results.

Nadia: And diagnostic three is where the mechanism class check comes in, using the parameter space signature rhoAT to see if the update behaves like what you claim it is, whether it’s shrinkage or something else.

Elias: And diagnostic four is about checking cross-task transfer, demanding that conditions like gamma P are greater than zero and the difference between gamma X and gamma P is less than or equal to two standard errors combined.

Priya: It sounds like a lot of checks that tie together statistical rigor with semantic meaning and operational consistency. It’s trying to prevent you from just claiming a defense when you haven't actually proven it generalizes properly.

Nadia: And the summary emphasizes that the resulting audit matrix shows that under strict coding, SafeLoRA fails all four diagnostics, which highlights how hard these standards are to meet in practice.

The paper's improvements: Elias: One of the main points they make is that this protocol is designed to reduce overclaiming. They want to ensure that a small gap reduction isn't being reported when the underlying mechanism isn't actually what it seems.

Nadia: And they suggest using these verdict labels—like near miss or missing evidence—instead of just giving a single score. That prevents people from reporting minor successes when they haven't met one of the four criteria, like failing the fresh subject threshold.

Priya: That’s crucial because it forces transparency about where the defense falls short. Instead of hiding that generalization failure in a single metric, you have to report it explicitly as a missing diagnostic.

Elias: They also introduce this concept of separating gap reduction from deployment accuracy cost. They want to make sure that utility loss isn't hidden inside the success metric itself, which means you have to report the deployment-accuracy cost outside of those gates too.

Nadia: So, they are pushing users to impose an additional utility floor on top of whatever gap reduction they find, making sure that the overall trade-off is what matters.

Priya: That makes sense. It shifts the focus from just achieving a low gap number to managing the trade-off between performance gain and deployment cost in a more structured way.

Elias: And they are also being careful about how they frame their tests, noting that cross architecture runs on models like Qwen2 point 5-1 point 5B-Instruct, Llama three 8B, and Phi three mini are just spot checks, not a proof that this standard scales universally <ref:2605.10575#pg2,Qwen2.5-1.5B-Instruct, Llama 3 8B, and Phi 3 mini>.

Nadia: So the improvement is less about making one perfect defense and more about creating a framework to evaluate *any* defense claim with enough evidence to be considered credible for reporting.

Conclusion: Nadia: To wrap up, the paper "Acceptance Cards: A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims" is proposing this rigorous four-diagnostic protocol—statistical reliability, semantic freshness, mechanism consistency, and cross-task transfer—as the standard for evaluating safe fine-tuning defense claims.

Elias: The key implication is that simply claiming a gap reduction isn't enough; you have to prove that reduction happens reliably and in multiple ways across different aspects of the model's behavior.

Priya: And they are providing tools—the Acceptance Cards, the documentation object, and the executable audit package—so researchers can actually execute this evaluation protocol on their own claims instead of just relying on a single number.

Nadia: They show that even with this strict standard applied to SafeLoRA on Gemma-two-2B-it, it fails the full card pass under strict mechanism coding, which is important context for anyone trying to deploy defenses right now <ref:2605.10575#pg1,on Gemma-2-2B-it>.

Elias: The paper also makes it clear that partial passes are reportable evidence, but missing diagnostics are considered missing evidence rather than implicit success on that gate.

Priya: So the final word is that these cards help prevent overclaiming and make sure we're reporting on real generalization failures, not just small numbers on a single metric.

Nadia: It’s a strict standard, but it seems satisfiable because each of those four gates targets a separate failure mode, so you can fix one problem at a time.

Elias: The paper suggests that the gap between different types of reduction—like the training distribution reduction versus the fresh domain reduction—is itself informative data.

Priya: It’s about detecting that generalization failure when it closes on trained topics without actually addressing the underlying routing mechanism, which is a very specific kind of problem.

Nadia: So, if you’re looking at a defense claim, don't just look at the gap reduction; use the Acceptance Cards to ensure you’ve checked for reliability and transferability.

Elias: That way, you get a clearer picture of what your defense actually achieves and where its real weaknesses lie.

Phongsakon Mark Konrad, Toygar Tanyel, Serkan Ayvaz

Centre for Industrial Software, University of Southern Denmark · ProMake

cs.CR, cs.AI, cs.LG

Submitted: 2026-05-11

Updated: 2026-11-01

Comments: Because the validation and scope of the four-diagnostic pass rule (deployment capability, cross-task criterion, mechanism diagnostic) and the breadth of the empirical audit do not support the claims as stated. The SafeLoRA result is a narrow out-of-domain stress test, not a judgment of the method. The work is being revised

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 91/100

The gist: Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss…

Key concepts

Acceptance Standard
This is the minimum requirement for a defense claim to be accepted. It demands that the fine-tuning recipe reduces the gap in a statistically reliable, semantically fresh, mechanism-consistent, and transferable manner. Passing this standard requires all four diagnostic gates to succeed strictly.
Diagnostic (a) - Statistical Reliability
This gate uses Question-Clustered Bootstrap to ensure the observed gap reduction is statistically sound. It requires the upper 95% confidence interval for the gap reduction ($\Delta$gap) to be below zero, confirming that the observed improvement is not due to random noise or finite-split artifacts.
Diagnostic (c) - Mechanism Alignment ($ ho_{AT}$)
This diagnostic checks if the fine-tuning mechanism aligns with its claimed class. It uses a parameter-space signature ($ ho_{AT}$) where shrinkage passes if $\rho_{AT} \ge 0.6$. This helps flag cases where an update claiming one behavior (like shrinkage) actually exhibits characteristics of another mechanism.
Diagnostic (d) - Cross-Task Transfer
This gate tests the generalization capability by requiring conditions like $\Gamma_P > 0$ and $|\Gamma_X - \Gamma_P| \le 2SE_{combined}$ to pass. This ensures the defense mechanism is robust enough to transfer its learned behavior successfully to a second, independent installable task.

Terminology

Summary

Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer <ref:2605. p. 0

Where the paper says it

ACCEPTANCE CARDS: A FOUR-DIAGNOSTIC STANDARD FOR SAFE FINE-TUNING DEFENSE CLAIMS A PREPRINT Phongsakon Mark Konrad*1 , Toygar Tanyel2 , and Serkan Ayvaz1 1Centre for Industrial Software, University of Southern Denmark, Alsion 2, Sønderborg, 6400, Denmark 2ProMake, Newark, DE, USA May 12, 2026 ABSTRACT Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims.

10575#pg2> The paper introduces Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims <ref:2605. p. 0
Where the paper says it

ACCEPTANCE CARDS: A FOUR-DIAGNOSTIC STANDARD FOR SAFE FINE-TUNING DEFENSE CLAIMS A PREPRINT Phongsakon Mark Konrad*1 , Toygar Tanyel2 , and Serkan Ayvaz1 1Centre for Industrial Software, University of Southern Denmark, Alsion 2, Sønderborg, 6400, Denmark 2ProMake, Newark, DE, USA May 12, 2026 ABSTRACT Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims.

10575#pg3> This protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass <ref:2605. p. 0
Where the paper says it

We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims. The protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass.

10575#pg4> The resulting audit on Gemma-2-2B-it shows that SafeLoRA fails the full-card pass under strict mechanism coding and still fails three of four under a permissive shrinkage relabel <ref:2605. p. 0
Where the paper says it

Re-scored under this installedgap protocol, SafeLoRA fails the full-card pass on Gemma-2-2B-it: under strict mechanism-class coding it fails all four diagnostics, and under a permissive shrinkage relabel it still fails three of four. This is a narrow installed-gap audit on one model family, not a global judgment of SafeLoRA’s effectiveness.

10575#pg5>

The Acceptance Standard

A minimum acceptance standard for this claim is defined as a fine-tuning-time recipe reducing an installed gap in a way that is statistically reliable, semantically fresh, mechanism-consistent, and transferable <ref:2605.10575#pg3> This standard requires four diagnostic gates to pass strictly for a full-card pass <ref:2605. p. 2

Where the paper says it

Satisfying the standard is evidence for the tested claim, not a deployability certificate: the card requires deployment-accuracy cost to be reported so that utility loss is not hidden inside the success metric, and a user may impose an additional utility floor on top of the card. 2 Four Diagnostic Gates Naive claim held-out gap smaller Acceptance card (a) clustered CI excludes no effect (b) fresh subjects clear signed SE (c) ρAT matches claimed class (d) transfer to installable task Accept only if all four pass 46-cell audit no strict pass closest: AC-AdamW Deployment accuracy: reported cost Missing diagnostic: not a pass Figure 1: The Acceptance Card standard.

10575#pg4> The four diagnostics are Diagnostic (a) catching finite-split and template-level noise, Diagnostic (b) catching subject- or prompt-split artifacts, Diagnostic (c) checking mechanism class consistency via the parameter-space signature ρAT, and Diagnostic (d) testing transfer to a second installable task <ref:2605. p. 1
Where the paper says it

Diagnostic Blocks Statistic Pass rule Does not prove (a) Bootstrap finite-split / template noise clustered CI for ∆gap upper 95% CI < 0 fresh-subject generalization (b) Fresh subjects subject-split artifacts paired and independent signed SE both upper 2SE checks < 0 mechanism or transfer (c) ρAT mechanism-label mismatch update projection ratio in slice claimed class matches signature efficacy (d) Transfer primary-task-only improvement ΓP vs ΓX w/ comb. SE installable cross-task gap; comparable reduction universal safety Conjunction overstated headline all four pass strictly (a)∧(b)∧(c)∧(d) deployment safety 2

10575#pg4> The conjunction rule requires all four diagnostics to pass strictly for a full-card pass <ref:2605. p. 1
Where the paper says it

The conjunction requires all four to pass strictly. Diagnostic Blocks Statistic Pass rule Does not prove (a) Bootstrap finite-split / template noise clustered CI for ∆gap upper 95% CI < 0 fresh-subject generalization (b) Fresh subjects subject-split artifacts paired and independent signed SE both upper 2SE checks < 0 mechanism or transfer (c) ρAT mechanism-label mismatch update projection ratio in slice claimed class matches signature efficacy (d) Transfer primary-task-only improvement ΓP vs ΓX w/ comb.

10575#pg4>

The Four Diagnostic Gates

The paper defines the thresholds for each diagnostic gate, which are conservative and designed to block specific over-readings <ref:2605.10575#pg4> Diagnostic (a) uses Question-Clustered Bootstrap, requiring the upper 95% CI on ∆gap to be below zero <ref:2605. p. 1

Where the paper says it

The conjunction requires all four to pass strictly. Diagnostic Blocks Statistic Pass rule Does not prove (a) Bootstrap finite-split / template noise clustered CI for ∆gap upper 95% CI < 0 fresh-subject generalization (b) Fresh subjects subject-split artifacts paired and independent signed SE both upper 2SE checks < 0 mechanism or transfer (c) ρAT mechanism-label mismatch update projection ratio in slice claimed class matches signature efficacy (d) Transfer primary-task-only improvement ΓP vs ΓX w/ comb.

10575#pg4> Diagnostic (b) requires both the paired and independent signed checks to hold, demanding that ∆gap + 2SEpaired Diagnostic (c) uses the parameter-space signature ρAT, where shrinkage passes if ρAT ≥ 0. p. 1
Where the paper says it

The conjunction requires all four to pass strictly. Diagnostic Blocks Statistic Pass rule Does not prove (a) Bootstrap finite-split / template noise clustered CI for ∆gap upper 95% CI < 0 fresh-subject generalization (b) Fresh subjects subject-split artifacts paired and independent signed SE both upper 2SE checks < 0 mechanism or transfer (c) ρAT mechanism-label mismatch update projection ratio in slice claimed class matches signature efficacy (d) Transfer primary-task-only improvement ΓP vs ΓX w/ comb.

6 and attack-targeted passes if ρAT Diagnostic (d) tests cross-task transfer, requiring conditions like ΓP > 0 and ΓX − ΓP ≤ 2SEcombined to pass <ref:2605. p. 1
Where the paper says it

Diagnostic Blocks Statistic Pass rule Does not prove (a) Bootstrap finite-split / template noise clustered CI for ∆gap upper 95% CI < 0 fresh-subject generalization (b) Fresh subjects subject-split artifacts paired and independent signed SE both upper 2SE checks < 0 mechanism or transfer (c) ρAT mechanism-label mismatch update projection ratio in slice claimed class matches signature efficacy (d) Transfer primary-task-only improvement ΓP vs ΓX w/ comb. SE installable cross-task gap; comparable reduction universal safety Conjunction overstated headline all four pass strictly (a)∧(b)∧(c)∧(d) deployment safety 2

10575#pg4>

Audit Results and Verdicts

The audit tests whether available defense evidence supports the full-card pass, not whether any recipe is deployable <ref:2605.10575#pg4> The audit results in a 46-cell matrix where no cell satisfies the strict conjunction for a full-card pass <ref:2605. p. 4

Where the paper says it

3 (full-card pass, near miss, missing evidence, undefined) keep these cases distinct. Across 46 audited cells, no cell with the required evidence satisfies the strict conjunction.

10575#pg5> The closest family is AC-AdamW, which passes (a) and (c) but fails (b) on the fresh-subject threshold and lacks a strict (d) pass <ref:2605. p. 0
Where the paper says it

In a 46-cell audit, no cell satisfies the strict conjunction. The closest family is a near miss that passes reliability and mechanism checks where the required data are available, but fails the fresh-subject threshold, lacks a strict transfer pass, and carries a measurable deployment-accuracy cost.

10575#pg5> The paper reports four verdict labels: full-card pass, near miss, missing evidence, and undefined <ref:2605. p. 1
Where the paper says it

2 Acceptance Standard This section types evidence for one claim: that a fine-tuning-time recipe reduces an installed evaluation-to-deployment gap. The standard defines four diagnostic gates, the threshold for each, a single conjunction rule, and four verdict labels (full-card pass, near miss, missing evidence, undefined) used to type cells that do not satisfy the strict conjunction.

10575#pg4>

Mechanism Diagnostic

The parameter-space signature ρAT is introduced as the new mechanism-class diagnostic among the four <ref:2605.10575#pg5> The pass rule for (c) states that a cell’s claimed class matches the signature in (3): shrinkage passes if ρAT ≥ 0. p. 3

Where the paper says it

(3) Pass rule. A cell’s claimed class matches the signature in (3): shrinkage passes if ρAT ≥ 0.

6, while attack-targeted passes if ρAT The diagnostic flags claimed attack-aware updates whose parameter-space signature aligns with AdamW shrinkage <ref:2605. p. 3
Where the paper says it

A pure shrinkage of the base gradient keeps ρAT near its baseline value; an attack-targeted update suppresses the projection on vA relative to vT and lowers ρAT . The diagnostic flags claimed attack-aware updates whose parameter-space signature aligns with AdamW shrinkage.

10575#pg5>

Limitations and Intended Use

The audit has empirical limits, as it is primarily based on Gemma-2-2B and uses cross-architecture runs as spot-checks rather than a scaling proof <ref:2605.10575#pg4> The standard is meant to reduce overclaiming and make partial results reportable, scoping SafeLoRA’s verdict to this installed-gap audit on Gemma-2-2B-it rather than a global judgment <ref:2605. p. 8

Where the paper says it

Broader Impacts. The standard is meant to reduce overclaiming and make partial results reportable; SafeLoRA’s verdict is scoped to this installed-gap audit on Gemma-2-2B-it, not to the method in general.

10575#pg5> The paper emphasizes that missing diagnostics block the full-card label but are not counted as empirical evidence of failure on that gate <ref:2605. p. 4
Where the paper says it

The 46-cell audit is an evidence-coverage audit rather than 46 complete four-gate failures. A missing or undefined gate blocks the full-card label but is not counted as empirical evidence that the method fails that gate; the four verdict labels in Section 2.

10575#pg5>

Conclusion

Safe fine-tuning defense claims should not rest on held-out gap reductions alone <ref:2605.10575#pg5> Acceptance Cards type evidence for one claim under four diagnostics, and bundle the protocol with a documentation object and an executable audit package <ref:2605. p. 8

Where the paper says it

7 Conclusion Safe fine-tuning defense claims should not rest on held-out gap reductions alone. Acceptance Cards type evidence for one claim under four diagnostics, and bundle the protocol with a documentation object and an executable audit package.

10575#pg5> Partial passes remain reportable evidence; missing diagnostics are missing evidence, not implicit success <ref:2605. p. 8
Where the paper says it

The closest family is a near miss with deployment-accuracy cost; SafeLoRA does not satisfy the full-card pass under the installed-gap protocol. Partial passes remain reportable evidence; missing diagnostics are missing evidence, not implicit success.

10575#pg5> The closest family is a near miss with deployment-accuracy cost, and SafeLoRA does not satisfy the full-card pass under the installed-gap protocol <ref:2605. p. 8
Where the paper says it

Acceptance Cards type evidence for one claim under four diagnostics, and bundle the protocol with a documentation object and an executable audit package. The closest family is a near miss with deployment-accuracy cost; SafeLoRA does not satisfy the full-card pass under the installed-gap protocol.

10575#pg5> The standard is strict but satisfiable, as each gate targets a separate failure mode <ref:2605. p. 6
Where the paper says it

9pp cost Strict conjunction (a)∧(b)∧(c)∧(d) near miss The closest partial result is a shrinkage-class recipe that drops gap by 5pp on fresh items at a cost of 12pp in deployment accuracy, without clearing the paired 2SE threshold on (b) and without a non-degenerate cross-task for (d). The standard is strict but satisfiable: each gate targets a separate failure mode, and nothing in the definitions prevents a future defense from passing all four.

10575#pg5> The gap between a 15. p. 0
Where the paper says it

ACCEPTANCE CARDS: A FOUR-DIAGNOSTIC STANDARD FOR SAFE FINE-TUNING DEFENSE CLAIMS A PREPRINT Phongsakon Mark Konrad*1 , Toygar Tanyel2 , and Serkan Ayvaz1 1Centre for Industrial Software, University of Southern Denmark, Alsion 2, Sønderborg, 6400, Denmark 2ProMake, Newark, DE, USA May 12, 2026 ABSTRACT Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims.

6pp training-distribution reduction and a 5. p. 6
Where the paper says it

The gap between a 15. 6pp training-distribution reduction and a 5.

2pp fresh-domain reduction is itself informative <ref:2605. p. 6
Where the paper says it

6pp training-distribution reduction and a 5. 2pp fresh-domain reduction is itself informative: the defense closes the gap on trained topics without addressing the underlying routing mechanism, exactly the generalization failure the standard is built to detect.

10575#pg5> The standard is built to detect the generalization failure that closes the gap on trained topics without addressing the underlying routing mechanism <ref:2605. p. 6
Where the paper says it

6pp training-distribution reduction and a 5. 2pp fresh-domain reduction is itself informative: the defense closes the gap on trained topics without addressing the underlying routing mechanism, exactly the generalization failure the standard is built to detect.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Precise evidence reporting for fine-tuning defenses can be achieved by using Acceptance Cards which require a recipe to pass four diagnostic gates: statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer. This protocol blocks overclaiming by requiring all four diagnostics to pass strictly before a full-card pass is awarded.

  2. A system can now rigorously audit defense claims by incorporating the parameter-space class-consistency signature rhoAT which classifies the mechanism class from the merged update without re-running the fine-tune. This allows researchers to identify cases where a published defense's update signs as shrinkage despite a non-shrinkage mechanism claim, as seen in SafeLoRA failing under strict mapping.

  3. The system can differentiate between partial evidence and full-card labels by using verdict labels such as near miss or missing evidence instead of relying on a single headline score. This prevents the reporting of small but real generalization when the fresh subject threshold is not met, as noted in the paper's Claim-Language Rules.

  4. Deployment safety can be assessed by separating gap reduction from deployment accuracy cost, ensuring that utility loss is not hidden inside the success metric. The audit requires reporting a deployment-accuracy cost of Xpp reported outside the gate, which allows users to impose an additional utility floor on top of the card.

  5. The system can be trained to recognize when cross-task transfer evidence is undefined by construction, such as when a baseline gap is within two standard errors of zero, and report this as N/A rather than pass. This prevents the conversion of missing evidence into success on the only diagnostic that tests transfer, which is a failure mode the standard is built to prevent.

Abstract

Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims. The protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass. Re-scored under this installed-gap protocol, SafeLoRA fails the full-card pass on Gemma-2-2B-it: under strict mechanism-class coding it fails all four diagnostics, and under a permissive shrinkage relabel it still fails three of four. This is a narrow installed-gap audit on one model family, not a global judgment of SafeLoRA's effectiveness. In a 46-cell audit, no cell satisfies the strict conjunction. The closest family is a near miss that passes reliability and mechanism checks where the required data are available, but fails the fresh-subject threshold, lacks a strict transfer pass, and carries a measurable deployment-accuracy cost.

Sources

Related papers