A Comprehensive Benchmark of Computational Models for Antibody--Antigen Binding Affinity

summary

The gist

AbBiBench introduces a benchmarking framework for antibody binding affinity maturation and design that treats the antibody–antigen (Ab–Ag) complex as the fundamental unit of evaluation…

In short

AbBiBench is a new framework that benchmarks computational models for antibody design by treating the entire antibody-antigen complex as the evaluation unit. It tests over 184,500 experimental measurements against 15 different protein models to find which ones best predict binding affinity and guide antibody redesign.

Key concepts

AbBiBench Framework
This is a unified system designed to evaluate antibody designs by considering the full antibody-antigen complex together. Instead of testing antibodies alone, it measures how well a model predicts binding within the context of both proteins, providing a rigorous standard for new AI models.
Inverse Folding Models
These are specific types of protein models that try to predict the original amino acid sequence given its 3D structure or binding properties. The study found that these models were the most accurate at predicting experimental binding affinities compared to other model types.
Binding Affinity Metrics
These are biophysical measurements used to quantify how strongly an antibody binds to an antigen. Key metrics include binding free energy (∆G), which measures the energy change of the interaction, and solvent-accessible surface areas (SASA) of key parts of the antibody.
Generative Utility
This refers to testing if a model can actually create new, functional antibody sequences or designs. The study used AbBiBench to test models on designing antibodies that bind to influenza H1N1, showing that structure-conditioned models were best at generating high-quality candidates.

Terminology used across episodes

This episode discusses

The paper

A Comprehensive Benchmark of Computational Models for Antibody--Antigen Binding Affinity · Read on arXiv

McWilliams School of Biomedical Informatics, UTHealth Houston · Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology · Texas Therapeutics Institute, Brown Foundation Institute of Molecular Medicine, UTHealth Houston

Computational antibody design requires benchmarks that measure functional binding rather than similarity to natural sequences. Here we present AbBiBench, which treats the antibody--antigen complex as the evaluation unit and harmonizes 216,560 experimental measurements spanning 13 antibodies and 9 antigens. We compare 20 computational approaches without task-specific training or fine-tuning. The analyses show that global structure-conditioned inverse folding yields the strongest and most consistent affinity rankings, whereas antibody-specific fine-tuning does not consistently improve performance. Across models, rankings are more useful for prioritizing likely affinity gains than for excluding weak variants. In a subsequent case study, multi-objective evaluation of model-generated F045-092 CDR-H3 variants for H1N1 binding revealed a trade-off between predicted binding potential and evolutionary sequence plausibility; all 21 selected candidates produced H1-reactive ELISA signals. AbBiBench provides an open framework for developing and evaluating function-aware antibody design methods.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: I'm Ines.

Marcus: And I'm Marcus, here with Yuki.

Ines: We're looking at the paper: "A Comprehensive Benchmark of Computational Models for Antibody--Antigen Binding Affinity".

Marcus: Here's the gist of the paper. AbBiBench introduces a benchmarking framework for antibody binding affinity maturation and design that treats the antibody–antigen (Ab–Ag) complex as the fundamental unit of evaluation <ref:2506.04235#pg2> This unified, biologically grounded evaluation framework facilitates the development of more effective, function-aware antibody design models <ref:2506.04235#pg2> Framework Introduction The AbBiBench framework is designed to move beyond traditional methods by evaluating antibodies based on their binding potential within the context of the full Ab–Ag complex <ref:2506.04235#pg2> The authors curate and standardize more than 184,500 experimental measurements of antibody mutants across 14 antibodies and 9 antigens to provide a rigorous benchmark for protein models <ref:2506.04235#pg2> This approach avoids the limitations of previous strategies that evaluate antibodies in isolation by focusing on how well a protein model scores the full Ab–Ag complex <ref:2506.04235#pg2> Model Comparison and Performance The study systematically compares 15 protein models, including masked language models, autoregressive language models, inverse folding models, diffusion-based generative models, and geometric graph models <ref:2506.04235#pg2> The evaluation involves two primary tasks: affinity prediction using model log-likelihood and antibody redesign <ref:2506.04235#pg2> Inverse folding models achieved the highest accuracy in predicting experimental binding affinity, attaining the highest average Spearman correlation across all tested models <ref:2506.04235#pg2> Generative Utility and Case Study To demonstrate its generative utility, AbBiBench was applied to antibody F045-092 to introduce binding to influenza H1N1 <ref:2506.04235#pg2> In this case study, models were evaluated based on sequence plausibility, structural integrity, and binding affinity metrics <ref:2506.04235#pg2> The results showed that structure-conditioned inverse folding models outperformed others in both affinity correlation and generation tasks <ref:2506.04235#pg2> Evaluation Metrics The computational evaluation metrics used include several biophysical measures such as binding free energy (∆G) and relative solvent-accessible surface areas (SASA) of epitope residues <ref:2506.04235#pg2> Other metrics assessed include sequence plausibility, structural integrity via AlphaFold pLDDT, and CDR-H3 loop structural deviation (cdrRMSD) <ref:2506.04235#pg2> Validation of Design In the case study for H1N1 influenza, the top 20% of Phase 1 variants selected by ESM-IF and SaProt demonstrated both strong binding potential and high sequence plausibility <ref:2506.04235#pg2> The final candidates were identified as those in the Pareto-optimal set across all five metrics, with ESM-IF achieving the greatest gain in affinity against the H1 subtype <ref:2506.04235#pg2> These results were further validated by performing in vitro ELISA binding assays on the designed mutants <ref:2506.04235#pg2> Limitations and Future Work Limitations noted include the lack of currently available experimental neutralization data (e.g., IC50) which restricts the ability of generative models to design therapeutic antibodies with tighter binding <ref:2506.04235#pg2> Future work plans include expanding the benchmark to incorporate additional therapeutic properties such as stability, immunogenicity, and developability <ref:2506.04235#pg2> The study also notes that some datasets are limited in size or derived from narrow mutational libraries, potentially impacting statistical power <ref:2506.04235#pg2> Data Curation Details The dataset compilation involved 14 studies covering various antigens including influenza, lysozyme, HER2, VEGF, integrin, Ang2, and SARS-CoV-2 <ref:2506.04235#pg2> Antibody libraries were constructed through phage or yeast display using deep mutational scanning or targeted positions to introduce mutations <ref:2506.04235#pg2> Experimental binding affinity measurements were standardized by taking the negative log for Kd and log for enrichment to ensure consistent directionality <ref:2506.04235#pg2> Model Specifics The comparison included various model categories such as Masked Language Models (e.g., ESM-2), Autoregressive PLMs (e.g., ProGen2), Inverse Folding Models (e.g., ProteinMPNN, ESM-IF), Diffusion-based generative models (e.g., DiffAb), and Geometric Graph Models (e.g., MEAN) <ref:2506.04235#pg2> Specific attention was paid to structure-informed models, such as those incorporating structural information into protein language models <ref:2506.04235#pg2> The authors also compared antibody-finetuned PLMs against general PLMs, noting varying impacts on zero-shot correlation <ref:2506.04235#pg2> Computational Metrics The study employed several computational metrics to assess the generated variants, including Binding Energy FoldX to quantify binding energy differences (∆∆G) <ref:2506.04235#pg2> Epitope SASA to measure changes in solvent-accessible surface area, and cdrDist and cdrRMSD for sequence similarity and structural deviation <ref:2506.04235#pg2> The affinity fold change was calculated by taking the difference between the pKd values of the mutant and wild-type antibodies <ref:2506.04235#pg2> Chain Order Insensitivity The analysis showed that providing antigen (mimicking SHM) and light chain context before the mutated heavy chain did not significantly improve zero-shot correlation for general autoregressive PLMs <ref:2506.04235#pg2> This finding suggests that autoregressive PLMs trained on single chain protein data remain largely insensitive to multimer chain ordering <ref:2506.

Paper summary: Ines: The core idea here is that this framework lets us evaluate antibody design by seeing how well a protein model predicts the binding potential when you put the whole antibody antigen complex into it. They curated over one hundred eighty-four thousand five hundred experimental measurements of antibody mutants across fourteen antibodies and nine antigens <ref:2506.04235#pg1,184,500 experimental measurements of antibody mutants across>.

Marcus: It claims this benchmark helps accelerate methods for antibody design that lead to clinical or diagnostic discoveries, specifically by focusing on how well a protein model scores that full Ab antigen complex.

Ines: They are testing a whole range of models here, from masked language models to autoregressive language models and inverse folding models and diffusion based generative models and geometric graph models.

Yuki: That’s a big variety of approaches they are comparing, which suggests they want to see if any one specific type of model is fundamentally better at this task than others.

Marcus: The paper says that when evaluating affinity prediction, inverse folding models achieved the highest accuracy in predicting experimental binding affinity, getting the highest average Spearman correlation across all the tested models.

Ines: That's a concrete result, showing that structure informed models like inverse folding have a strong performance here compared to the others they tested.

Yuki: From my perspective, if these computational tools can reliably predict affinity based on this complex unit, it means we might be able to propose variants much faster than relying only on slow experimental methods like traditional screening.

Conclusion: Ines: So, to wrap up this piece on "A Comprehensive Benchmark of Computational Models for Antibody--Antigen Binding Affinity," the authors are establishing AbBiBench as the standard way to benchmark antibody design models by using the full antibody antigen complex as their evaluation unit.

Marcus: They found that inverse folding models performed best in predicting experimental binding affinity, and they showed that when applied to a case study with influenza H1N1, structure conditioned inverse folding models outperformed others in both affinity correlation and the generation tasks.

Ines: What this means for us is that we now have a rigorous way to test whether these machine learning models are actually learning the right biological rules for how antibodies bind antigens, rather than just memorizing sequences.

Yuki: It suggests that if these models can be reliably predicted to work well on this benchmark, we gain confidence in using them to propose new antibody variants that might have genuinely improved binding potential.

Marcus: The paper notes some limitations though, specifically mentioning that they lack currently available experimental neutralization data, like the IC50 values, which limits how much generative models can design antibodies with tighter binding.

Ines: They also pointed out that some datasets are limited in size or come from narrow mutational libraries, which means the statistical power for those specific studies might be weaker than what they hope for.

Yuki: So, while this framework is solid for testing prediction and generation based on affinity metrics and sequence plausibility, we still need more data on things like stability or immunogenicity before we can fully trust these designs in a real therapeutic context.

More episodes

← Home