A Comprehensive Benchmark of Computational Models for Antibody--Antigen Binding Affinity
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: I'm Ines.
Marcus: And I'm Marcus, here with Yuki.
Ines: We're looking at the paper: "A Comprehensive Benchmark of Computational Models for Antibody--Antigen Binding Affinity".
Marcus: Here's the gist of the paper. AbBiBench introduces a benchmarking framework for antibody binding affinity maturation and design that treats the antibody–antigen (Ab–Ag) complex as the fundamental unit of evaluation <ref:2506.04235#pg2> This unified, biologically grounded evaluation framework facilitates the development of more effective, function-aware antibody design models <ref:2506.04235#pg2> Framework Introduction The AbBiBench framework is designed to move beyond traditional methods by evaluating antibodies based on their binding potential within the context of the full Ab–Ag complex <ref:2506.04235#pg2> The authors curate and standardize more than 184,500 experimental measurements of antibody mutants across 14 antibodies and 9 antigens to provide a rigorous benchmark for protein models <ref:2506.04235#pg2> This approach avoids the limitations of previous strategies that evaluate antibodies in isolation by focusing on how well a protein model scores the full Ab–Ag complex <ref:2506.04235#pg2> Model Comparison and Performance The study systematically compares 15 protein models, including masked language models, autoregressive language models, inverse folding models, diffusion-based generative models, and geometric graph models <ref:2506.04235#pg2> The evaluation involves two primary tasks: affinity prediction using model log-likelihood and antibody redesign <ref:2506.04235#pg2> Inverse folding models achieved the highest accuracy in predicting experimental binding affinity, attaining the highest average Spearman correlation across all tested models <ref:2506.04235#pg2> Generative Utility and Case Study To demonstrate its generative utility, AbBiBench was applied to antibody F045-092 to introduce binding to influenza H1N1 <ref:2506.04235#pg2> In this case study, models were evaluated based on sequence plausibility, structural integrity, and binding affinity metrics <ref:2506.04235#pg2> The results showed that structure-conditioned inverse folding models outperformed others in both affinity correlation and generation tasks <ref:2506.04235#pg2> Evaluation Metrics The computational evaluation metrics used include several biophysical measures such as binding free energy (∆G) and relative solvent-accessible surface areas (SASA) of epitope residues <ref:2506.04235#pg2> Other metrics assessed include sequence plausibility, structural integrity via AlphaFold pLDDT, and CDR-H3 loop structural deviation (cdrRMSD) <ref:2506.04235#pg2> Validation of Design In the case study for H1N1 influenza, the top 20% of Phase 1 variants selected by ESM-IF and SaProt demonstrated both strong binding potential and high sequence plausibility <ref:2506.04235#pg2> The final candidates were identified as those in the Pareto-optimal set across all five metrics, with ESM-IF achieving the greatest gain in affinity against the H1 subtype <ref:2506.04235#pg2> These results were further validated by performing in vitro ELISA binding assays on the designed mutants <ref:2506.04235#pg2> Limitations and Future Work Limitations noted include the lack of currently available experimental neutralization data (e.g., IC50) which restricts the ability of generative models to design therapeutic antibodies with tighter binding <ref:2506.04235#pg2> Future work plans include expanding the benchmark to incorporate additional therapeutic properties such as stability, immunogenicity, and developability <ref:2506.04235#pg2> The study also notes that some datasets are limited in size or derived from narrow mutational libraries, potentially impacting statistical power <ref:2506.04235#pg2> Data Curation Details The dataset compilation involved 14 studies covering various antigens including influenza, lysozyme, HER2, VEGF, integrin, Ang2, and SARS-CoV-2 <ref:2506.04235#pg2> Antibody libraries were constructed through phage or yeast display using deep mutational scanning or targeted positions to introduce mutations <ref:2506.04235#pg2> Experimental binding affinity measurements were standardized by taking the negative log for Kd and log for enrichment to ensure consistent directionality <ref:2506.04235#pg2> Model Specifics The comparison included various model categories such as Masked Language Models (e.g., ESM-2), Autoregressive PLMs (e.g., ProGen2), Inverse Folding Models (e.g., ProteinMPNN, ESM-IF), Diffusion-based generative models (e.g., DiffAb), and Geometric Graph Models (e.g., MEAN) <ref:2506.04235#pg2> Specific attention was paid to structure-informed models, such as those incorporating structural information into protein language models <ref:2506.04235#pg2> The authors also compared antibody-finetuned PLMs against general PLMs, noting varying impacts on zero-shot correlation <ref:2506.04235#pg2> Computational Metrics The study employed several computational metrics to assess the generated variants, including Binding Energy FoldX to quantify binding energy differences (∆∆G) <ref:2506.04235#pg2> Epitope SASA to measure changes in solvent-accessible surface area, and cdrDist and cdrRMSD for sequence similarity and structural deviation <ref:2506.04235#pg2> The affinity fold change was calculated by taking the difference between the pKd values of the mutant and wild-type antibodies <ref:2506.04235#pg2> Chain Order Insensitivity The analysis showed that providing antigen (mimicking SHM) and light chain context before the mutated heavy chain did not significantly improve zero-shot correlation for general autoregressive PLMs <ref:2506.04235#pg2> This finding suggests that autoregressive PLMs trained on single chain protein data remain largely insensitive to multimer chain ordering <ref:2506.
Paper summary: Ines: The core idea here is that this framework lets us evaluate antibody design by seeing how well a protein model predicts the binding potential when you put the whole antibody antigen complex into it. They curated over one hundred eighty-four thousand five hundred experimental measurements of antibody mutants across fourteen antibodies and nine antigens <ref:2506.04235#pg1,184,500 experimental measurements of antibody mutants across>.
Marcus: It claims this benchmark helps accelerate methods for antibody design that lead to clinical or diagnostic discoveries, specifically by focusing on how well a protein model scores that full Ab antigen complex.
Ines: They are testing a whole range of models here, from masked language models to autoregressive language models and inverse folding models and diffusion based generative models and geometric graph models.
Yuki: That’s a big variety of approaches they are comparing, which suggests they want to see if any one specific type of model is fundamentally better at this task than others.
Marcus: The paper says that when evaluating affinity prediction, inverse folding models achieved the highest accuracy in predicting experimental binding affinity, getting the highest average Spearman correlation across all the tested models.
Ines: That's a concrete result, showing that structure informed models like inverse folding have a strong performance here compared to the others they tested.
Yuki: From my perspective, if these computational tools can reliably predict affinity based on this complex unit, it means we might be able to propose variants much faster than relying only on slow experimental methods like traditional screening.
Conclusion: Ines: So, to wrap up this piece on "A Comprehensive Benchmark of Computational Models for Antibody--Antigen Binding Affinity," the authors are establishing AbBiBench as the standard way to benchmark antibody design models by using the full antibody antigen complex as their evaluation unit.
Marcus: They found that inverse folding models performed best in predicting experimental binding affinity, and they showed that when applied to a case study with influenza H1N1, structure conditioned inverse folding models outperformed others in both affinity correlation and the generation tasks.
Ines: What this means for us is that we now have a rigorous way to test whether these machine learning models are actually learning the right biological rules for how antibodies bind antigens, rather than just memorizing sequences.
Yuki: It suggests that if these models can be reliably predicted to work well on this benchmark, we gain confidence in using them to propose new antibody variants that might have genuinely improved binding potential.
Marcus: The paper notes some limitations though, specifically mentioning that they lack currently available experimental neutralization data, like the IC50 values, which limits how much generative models can design antibodies with tighter binding.
Ines: They also pointed out that some datasets are limited in size or come from narrow mutational libraries, which means the statistical power for those specific studies might be weaker than what they hope for.
Yuki: So, while this framework is solid for testing prediction and generation based on affinity metrics and sequence plausibility, we still need more data on things like stability or immunogenicity before we can fully trust these designs in a real therapeutic context.
McWilliams School of Biomedical Informatics, UTHealth Houston · Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology · Texas Therapeutics Institute, Brown Foundation Institute of Molecular Medicine, UTHealth Houston
q-bio.QM, cs.AI, cs.CE, cs.LG, q-bio.BM
Submitted: 2025-05-23
Updated: 2026-11-03
Code: https://github.com/MSBMI-SAFE/AbBiBench
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 75/100
The gist: AbBiBench introduces a benchmarking framework for antibody binding affinity maturation and design that treats the antibody–antigen (Ab–Ag) complex as the fundamental unit of evaluation…
Key concepts
- AbBiBench Framework
- This is a unified system designed to evaluate antibody designs by considering the full antibody-antigen complex together. Instead of testing antibodies alone, it measures how well a model predicts binding within the context of both proteins, providing a rigorous standard for new AI models.
- Inverse Folding Models
- These are specific types of protein models that try to predict the original amino acid sequence given its 3D structure or binding properties. The study found that these models were the most accurate at predicting experimental binding affinities compared to other model types.
- Binding Affinity Metrics
- These are biophysical measurements used to quantify how strongly an antibody binds to an antigen. Key metrics include binding free energy (∆G), which measures the energy change of the interaction, and solvent-accessible surface areas (SASA) of key parts of the antibody.
- Generative Utility
- This refers to testing if a model can actually create new, functional antibody sequences or designs. The study used AbBiBench to test models on designing antibodies that bind to influenza H1N1, showing that structure-conditioned models were best at generating high-quality candidates.
Terminology
Summary
AbBiBench introduces a benchmarking framework for antibody binding affinity maturation and design that treats the antibody–antigen (Ab–Ag) complex as the fundamental unit of evaluation <ref:2506. p. 0 ABBIBENCH: A BENCHMARK FOR ANTIBODY BINDING AFFINITY MATURATION AND DESIGN
Xinyan Zhao1
, Yi-Ching Tang1
, Akshita Singh1
, Victor J Cantu1
, KwanHo An1
, Junseok Lee∗2
, Adam
E Stogsdill1
, Ibraheem M Hamdi1
, Ashwin Kumar Ramesh3
, Zhiqiang An3
, Xiaoqian Jiang1
, and
Yejin Kim1
1McWilliams School of Biomedical Informatics, UTHealth Houston
2Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and
Technology
3Texas Therapeutics Institute, Brown Foundation Institute of Molecular Medicine, UTHealth
Houston
ABSTRACT
We introduce AbBiBench (Antibody Binding Benchmarking), a benchmarking
framework for antibody binding affinity maturation and design. Unlike previous
strategies that evaluate antibodies in isolation, typically by comparing them to natural sequences with metrics such as amino acid recovery rate or structural RMSD,
AbBiBench instead treats the antibody–antigen (Ab–Ag) complex as the fundamental unit. Our findings show that structure-conditioned inverse folding models outperform others in both affinity correlation and generation
tasks. Overall, AbBiBench provides a unified, biologically grounded evaluation
framework to facilitate the development of more effective, function-aware antibody
design models.Where the paper says it
Where the paper says it
Framework Introduction
The AbBiBench framework is designed to move beyond traditional methods by evaluating antibodies based on their binding potential within the context of the full Ab–Ag complex <ref:2506.04235#pg2> The authors curate and standardize more than 184,500 experimental measurements of antibody mutants across 14 antibodies and 9 antigens to provide a rigorous benchmark for protein models <ref:2506. p. 0 We first curate, standardize,
and share more than 184,500 experimental measurements of antibody mutants
across 14 antibodies and 9 antigens—including influenza, lysozyme, HER2, VEGF,
integrin, Ang2, and SARS-CoV-2—covering both heavy-chain and light-chain
mutations. Using these datasets, we systematically compare 15 protein models
including masked language models, autoregressive language models, inverse folding models, diffusion-based generative models, and geometric graph models by
comparing the correlation between model likelihood and experimental affinity
values. Unlike previous
strategies that evaluate antibodies in isolation, typically by comparing them to natural sequences with metrics such as amino acid recovery rate or structural RMSD,
AbBiBench instead treats the antibody–antigen (Ab–Ag) complex as the fundamental unit. It evaluates an antibody design’s binding potential by measuring how
well a protein model scores the full Ab–Ag complex.Where the paper says it
Where the paper says it
Model Comparison and Performance
The study systematically compares 15 protein models, including masked language models, autoregressive language models, inverse folding models, diffusion-based generative models, and geometric graph models <ref:2506.04235#pg2> The evaluation involves two primary tasks: affinity prediction using model log-likelihood and antibody redesign <ref:2506. p. 5 3 EVALUATION TASKS
Our benchmark comprises (i) zero-shot affinity prediction using retrospective experimental affinity
data and (ii) antibody generation by sampling from the models. 1 ZERO-SHOT PREDICTION OF EXPERIMENTAL BINDING AFFINITY USING MODEL
LOG-LIKELIHOOD
To measure how well a model’s zero-shot predicted log-likelihood aligns with wet-lab verified
affinity, we calculated the Spearman correlation between the model likelihood and experimentally
measured binding affinity. 4 RESULTS
4. 1 ZERO-SHOT PREDICTION OF EXPERIMENTAL BINDING AFFINITY USING MODEL
LOG-LIKELIHOOD
As a result, inverse folding models achieved the highest accuracy in predicting experimental binding
affinity, attaining the highest average Spearman correlation (Fig.Where the paper says it
Where the paper says it
Generative Utility and Case Study
To demonstrate its generative utility, AbBiBench was applied to antibody F045-092 to introduce binding to influenza H1N1 <ref:2506.04235#pg2> In this case study, models were evaluated based on sequence plausibility, structural integrity, and binding affinity metrics <ref:2506. p. 2 All models are evaluated on two tasks: affinity prediction
and antibody redesign. Five computational metrics assess newly designed antibodies from sequence
plausibility, structural integrity, and binding affinity perspectives. We sample
new antibody variants with the top-performing models, rank them by the structural
integrity and biophysical properties of the Ab–Ag complex, and assess them with
in vitro ELISA binding assays. Our findings show that structure-conditioned inverse folding models outperform others in both affinity correlation and generation
tasks.Where the paper says it
Where the paper says it
Evaluation Metrics
The computational evaluation metrics used include several biophysical measures such as binding free energy (∆G) and relative solvent-accessible surface areas (SASA) of epitope residues <ref:2506.04235#pg2> Other metrics assessed include sequence plausibility, structural integrity via AlphaFold pLDDT, and CDR-H3 loop structural deviation (cdrRMSD) <ref:2506. p. 6 We also examined the diversity of the selected antibody
designs. We compared antibody PLM embeddings (AntiBERTy), sequence similarity (cdrDist),
and structural deviation of the CDR-H3 loop (cdrRMSD) relative to the wild type (Supplement 5).Where the paper says it
Validation of Design
In the case study for H1N1 influenza, the top 20% of Phase 1 variants selected by ESM-IF and SaProt demonstrated both strong binding potential and high sequence plausibility <ref:2506.04235#pg2> The final candidates were identified as those in the Pareto-optimal set across all five metrics, with ESM-IF achieving the greatest gain in affinity against the H1 subtype <ref:2506. p. 6 In
Phase 2, we generated full Ab-Ag complex structures with AlphaFold 3 for this subset and computed
pLDDT, epitope SASA, and inverse folding likelihood. We identified the final candidates as those in
the Pareto-optimal set across all five metrics. These 21
variants have been successfully expressed in vitro, and conduct ELISA assays to quantify their binding affinity to H1N1 hemagglutinin. The results of these assays directly validate the computational
predictions and help assess the true binding potential of the designed antibodies.Where the paper says it
Where the paper says it
Limitations and Future Work
Limitations noted include the lack of currently available experimental neutralization data (e.g., IC50) which restricts the ability of generative models to design therapeutic antibodies with tighter binding <ref:2506. p. 3 We are specifically
interested in interactions that involve an antibody’s CDR loops and an antigen and for this reason,
only 6 Ab-Ag pairs out of BindgingGYM’s 30 overlap with those we have curated here. 3 ABBIBENCH: AN ANTIBODY BINDING BENCHMARK
We introduce AbBiBench, a benchmark to evaluate protein models’ ability to predict and design
high-affinity antibodies. Limitations of this
study include the lack of currently available experimental neutralization data and the relatively small
size of some benchmark datasets (Supplement 8). In future work, we plan to expand the benchmark
to include additional therapeutic properties such as stability, immunogenicity, and developability, and
to incorporate functional assay data (e. We only included datasets with at least 20 mutated antibodies to ensure the statistical
significance of evaluation. In each binding affinity study, antibody libraries are constructed through phage or yeast display,
introducing mutations via deep mutational scanning or at targeted positions.Where the paper says it
Where the paper says it
Where the paper says it
Data Curation Details
The dataset compilation involved 14 studies covering various antigens including influenza, lysozyme, HER2, VEGF, integrin, Ang2, and SARS-CoV-2 <ref:2506.04235#pg2> Antibody libraries were constructed through phage or yeast display using deep mutational scanning or targeted positions to introduce mutations <ref:2506. p. 3 In each binding affinity study, antibody libraries are constructed through phage or yeast display,
introducing mutations via deep mutational scanning or at targeted positions. Some libraries were
designed computationally using machine learning Li et al. (2006), resulting in variant sets ranging from 67 to 65,535. For
consistency, we standardized experimental affinity measurements by taking the negative log for Kd
and log for enrichment, so that higher values indicate stronger binding (Supplement 2, Table S1).Where the paper says it
Where the paper says it
Model Specifics
The comparison included various model categories such as Masked Language Models (e.g., ESM-2), Autoregressive PLMs (e. p. 4 (2022a) antibody data while preserving pre-trained knowledge
on general proteins. Autoregressive Protein Language Models: Unlike MLMs, which predict masked tokens based on
bidirectional context, autoregressive PLMs generate the next token using only left-to-right context. Inverse folding models are also autoregressive but based on global structure embeddings. ProGen2 and ProtGPT2 can serve as structure-agnostic comparisons for inverse folding models. (2025),
based on ESM-IF1, is fine-tuned on antibody structures from SAbDab and OAS. Diffusion-Based Generative Models: Diffusion models approach antibody design as a denoising
process, transforming Gaussian noise into a target antibody by learning the joint distribution of
atomic coordinates, orientations, and residue identities. We first curate, standardize,
and share more than 184,500 experimental measurements of antibody mutants
across 14 antibodies and 9 antigens—including influenza, lysozyme, HER2, VEGF,
integrin, Ang2, and SARS-CoV-2—covering both heavy-chain and light-chain
mutations. Using these datasets, we systematically compare 15 protein models
including masked language models, autoregressive language models, inverse folding models, diffusion-based generative models, and geometric graph models by
comparing the correlation between model likelihood and experimental affinity
values. (1994); Kohler & ¨
∗Work done while at University of Texas Health Science Center. 1
arXiv:2506. Recognizing the limitations of
traditional computational evaluation metrics, our approach explicitly incorporates antibody-antigen
complex information, thereby aligning more closely with the biological realities of antibody interactions. Our results demonstrate that global structure-informed protein language model used for inverse
folding methods, such as ESM-IF and ProteinMPNN, outperform other evaluated computational
models, primarily due to their effective integration of structural context. Interestingly, we also
found that antibody-finetuned PLMs have varying impacts on zero-shot correlation to binding affinity. When we compare general PLM ESM-2 to its Ab-finetuned model CurrAb Burbach & Briney
(2025), CurrAb improved Spearman correlation by +0.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Computational Metrics
The study employed several computational metrics to assess the generated variants, including Binding Energy FoldX to quantify binding energy differences (∆∆G) <ref:2506.04235#pg2> Epitope SASA to measure changes in solvent-accessible surface area, and cdrDist and cdrRMSD for sequence similarity and structural deviation <ref:2506. p. 6 We also
computed inverse folding likelihoods using ProteinMPNN to determine whether each sequence is
compatible with its backbone structure—higher scores indicate greater foldability. Binding potential
was evaluated using three structure-based metrics: (1) binding free energy (∆G) to estimate Ab–Ag
interaction strength, (2) epitope SASA to measure differences in the solvent-accessible antigen
surface area upon binding, and (3) complex structure confidence (AlphaFold pLDDT Abramson et al. To
evaluate how effectively a model prioritizes the most promising antibodies, we also reported 5-fold
precision@10 – the proportion of top 10 ranked variants that achieve at least 5-fold improvement
in binding affinity compared to the wild type. The calculation of affinity fold change is detailed
in Supplement 5.Where the paper says it
Where the paper says it
Chain Order Insensitivity
The analysis showed that providing antigen (mimicking SHM) and light chain context before the mutated heavy chain did not significantly improve zero-shot correlation for general autoregressive PLMs <ref:2506.04235#pg2> This finding suggests that autoregressive PLMs trained on single chain protein data remain largely insensitive to multimer chain ordering <ref:2506. p. 3 (2021) is a 12-layer BERT model trained on 57 million heavy- and light-chain sequences
from antibody database (OAS Olsen et al. Incorporating structural information into
protein language models (PLMs) improves their ability to capture spatial context beyond sequence
proximity.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Integration of AbBiBench into antibody design models allows for evaluation of protein models by
measuring how well a protein model scores the full Ab–Ag complex
instead of relying on metrics likeamino acid recovery rate or structural RMSD.
This enables models to be directly compared based on how well they score thefull Ab–Ag complex,
providing a more biologically grounded assessment. -
Development of structure-conditioned inverse folding models can be used to predict sequence mutations that
maintain or improve structural integrity
because these models are shown to achieve the highest accuracy in predicting experimental binding affinity, suggesting they can generate sequences that are structurally stable and high-affinity. -
Implementation of zero-shot prediction via model log likelihood correlation allows for identifying
affinity-enhancing mutations in a zero-shot setting,
enabling antibody design models to prioritize promising variants based on how well their predicted likelihood aligns with wet lab verified binding affinity. -
Application of generative models like DiffAb and MEAN can be used to
design new antibody variants with stronger H1N1 influenza affinity
by sampling from the model's learned distribution, allowing for the generation of sequences that achieve amean pLDDT of 83.47
in predicted complexes and show again of H1 binding activity.
-
Use of graph-based CDR imputation models like MEAN and dyMEAN allows for the joint prediction of
amino acid sequence and full-atom structure (backbone + sidechains) of masked CDR regions,
enabling the design of antibodies by generating both sequence identity and precise 3D coordinates simultaneously under geometric constraints.
Abstract
Computational antibody design requires benchmarks that measure functional binding rather than similarity to natural sequences. Here we present AbBiBench, which treats the antibody--antigen complex as the evaluation unit and harmonizes 216,560 experimental measurements spanning 13 antibodies and 9 antigens. We compare 20 computational approaches without task-specific training or fine-tuning. The analyses show that global structure-conditioned inverse folding yields the strongest and most consistent affinity rankings, whereas antibody-specific fine-tuning does not consistently improve performance. Across models, rankings are more useful for prioritizing likely affinity gains than for excluding weak variants. In a subsequent case study, multi-objective evaluation of model-generated F045-092 CDR-H3 variants for H1N1 binding revealed a trade-off between predicted binding potential and evolutionary sequence plausibility; all 21 selected candidates produced H1-reactive ELISA signals. AbBiBench provides an open framework for developing and evaluating function-aware antibody design methods.
Sources
- Inverse folding for antibody sequence design using deep learning
- Learning from Protein Structure with Geometric Vector Perceptrons
- Conditional Antibody Design as 3D Equivariant Graph Translation
- End-to-End Full-Atom Antibody Design
- Deciphering antibody affinity maturation with language models and weakly supervised learning
Related papers
- ViroGym: Realistic Large-Scale Benchmarks for Evaluating Viral Proteins
- CellCanon: An Agentic Cellular World Model for Prediction, Explanation, and Observation-Driven Updating
- Probing and steering biology across Boltz-1s trunk-diffusion boundary
- Trajectory Inference of Human Aging from Cross-Sectional DNA Methylation Data
- Bridging the Modality Bottleneck in Pathology MIL through Molecularly Guided Feature Adaptation
- Molecular notation can act as a shortcut in AI benchmark evaluation