DeepDiff: Neuro-Dynamic Regression-Freedom Checking for Security Patch Validation

arXiv:2610.39066 · cs.CR, cs.SE · Submitted 2026-10-28 · Read on arXiv

cs.CR, cs.SE

Submitted: 2026-10-28

Updated: 2026-10-28

Comments: 22 pages, 4 figures

License: http://creativecommons.org/licenses/by/4.0/

The gist: Automatically validating AI-generated security patches remains a significant challenge.

Abstract

Automatically validating AI-generated security patches remains a significant challenge. Eliminating a reported crash and passing existing test suites do not provide sufficient confidence that a patch correctly addresses the underlying vulnerability without introducing regressions. We present DeepDiff, a technique for regression-freedom checking that differentially evaluates candidate patches against the original vulnerable program using non-crashing inputs that exercise the affected code paths. The expectation of regression-freedom is that a correct patch should eliminate the vulnerability triggering crash while preserving the intended behavior on non-crashing inputs. To support this evaluation, we introduce a dataset of correct and incorrect patch candidates for open-source memory-safety vulnerabilities from CyberGym. We explore different approaches for non-triggering test input generation to reach the patched code, a purely neural regression-freedom checker, and a hybrid approach that combines dynamic analysis with neural techniques. Our experimental results demonstrate the effectiveness of regression-freedom checking in validating security patches while outlining the challenges that remain.

Related papers