Towards Robust Numerical Claim Verification

arXiv:2610.00689 · cs.CL · Submitted 2026-09-30 · Read on arXiv

cs.CL

Submitted: 2026-09-30

Updated: 2026-09-30

Comments: Accepted to AACL-IJCNLP 2026 Findings

Code: https://github.com/iai-group/numpert-fine-tune

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large language models (LLMs) are widely used for claim verification, yet remain brittle for numerical reasoning: even small changes in value can sharply degrade accuracy.

Terminology

Abstract

Large language models (LLMs) are widely used for claim verification, yet remain brittle for numerical reasoning: even small changes in value can sharply degrade accuracy. We show that this brittleness persists in frontier LLMs, but can be mitigated through adversarial fine-tuning on numerically perturbed examples. Using parameter-efficient fine-tuning, small Qwen3 models (0.6B 8B) reach 98.7% accuracy on label-flipping perturbations, outperforming larger zero-shot models and frontier systems (GPT-5.4 Pro (74.0%) and Gemini 2.5 Flash (73.9%)). The gains generalise to unseen perturbation types, indicating robust numerical decision boundaries rather than memorised edits. Robustness also transfers without target-domain data, significantly improving cross-lingual performance in Spanish. We further show that the same fine-tuning recipe confers robustness to evidence-side perturbations, using the VitaminC dataset.

Related papers