QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space".
Jane: Here's the gist of the paper. Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property prediction, but their evaluation as machine learning targets has been complicated by the absence of a principled out-of-distribution evaluation protocol at the atomic level.
Tom: In this work, we propose a held-out evaluation protocol that clusters atomic environments by SOAP descriptors and computes metrics accounting only for cluster labels unseen during training.
Jane: Following this procedure, we use 5×5 cross-validation and Tukey’s HSD to run a statistically rigorous comparison of E(3)-equivariant against non-equivariant, rotationally augmented models for predicting electron populations and multipoles of H, C, N, and O atoms.
Tom: Building on our results, we introduce the Quantum Topological Neural Network (QT-Net), a rotationally augmented, non-equivariant graph neural network.
Title and authors: Tom: So we’re diving into the paper "QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space." What does that title actually tell us about what they are trying to fix?
Jane: It means they’re looking at how we test these models for atomic properties and saying the way we currently test them is fundamentally broken when it comes to out-of-distribution generalization.
Lu: They point out a major pitfall in existing literature where the splits used for testing don't isolate entirely new atomic environments, which they call that key problem > Page three <ref:2605.10458#pg3>.
Meng: So instead of just randomly picking molecules to test, they’ve put together a specific strategy to make sure the test atoms are from environments the model has never seen before > Page one <ref:2605.10458#pg1>.
Lalam: They are using this clustering idea based on SOAP descriptors to create a held-out test set where those cluster labels never appear in the training data > Page three <ref:2605.10458#pg3>.
Tom: And this protocol lets them calculate metrics that only consider those cluster labels unseen during training, which they claim gives a more faithful out-of-distribution evaluation > Page one <ref:2605.10458#pg1>.
Jane: It’s about making sure the performance numbers we report aren't just lucky because the test set happened to be easy for the model > Page one <ref:2605.10458#pg1>.
Lu: They use this structured testing approach with five by five cross-validation and Tukey’s honestly significant difference framework to compare their equivariant and non-equivariant models statistically, which is a rigorous way to benchmark them > Page one <ref:2605.10458#pg1>.
Meng: I mean, it sounds like they are trying to stop the kind of overconfident results that happen when you just look at all the atoms in a test set without any regard for where those atoms came from > Page one <ref:2605.10458#pg1>.
Lalam: The paper shows that this rethinking of how we evaluate is what lets them draw more meaningful conclusions about how these models perform across different chemical spaces > Page one <ref:2605.10458#pg1>.
Tom: So, to summarize the title and authors of "QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space," it’s proposing a new way to evaluate atomic properties that directly addresses the problem of out-of-distribution generalization in molecular property prediction > Page one <ref:2605.10458#pg1>.
Jane: They show how atomic properties like partial charges or multipoles give us chemically meaningful information that can actually inform downstream predictions, but they argue we haven't had a proper way to evaluate those targets yet > Page one <ref:2605.10458#pg1>, chemically meaningful information that can inform.
Title and authors: Lu: The main thing they introduce is QT-Net, which is a rotationally augmented and non-equivariant graph neural network built specifically to learn these quantum topological atomic properties > Page one <ref:2605.10458#pg1>.
Meng: They built this model by including loss functions that track electron population, localization index lambda, atomic dipole moment mu, and the quadrupole moment Q > Page two <ref:2605.10458#pg2>.
Lalam: And they show that using QT-Net lets you infer properties of atoms in molecules from QM9 outside their training set > Page one <ref:2605.10458#pg1>.
Tom: That’s a pretty big claim, so they are suggesting that these inferred properties can actually improve downstream molecular property prediction when you use them as input features > Page one <ref:2605.10458#pg1>.
Jane: And they validate this by showing that the dipole moments calculated from QT-Net outputs recover the ground truth values reported in QM9 > Page one <ref:2605.10458#pg1>.
Lu: This suggests that the physical consistency of these learned properties is high enough to be useful for other tasks, which is a pretty strong result when you think about representation learning > Page one <ref:2605.10458#pg1>.
Tom: So, what they are doing is creating a way to test atomic property models that doesn't rely on the flawed sampling methods we’ve seen before > Page one <ref:2605.10458#pg1>, QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space>.
Jane: They introduce the QT-Net model and prove it can infer properties for atoms outside their training set and that those inferred properties actually help improve other predictions > Page one <ref:2605.10458#pg1>.
Lu: The main implication is that the challenge has shifted from how to model these targets to which targets you choose to model and on what chemistry matters most > Page one <ref:2605.10458#pg1>.
Meng: From a production standpoint, it means we need better ways of knowing when a model is truly generalizing and not just overfit on the specific molecular space it was shown > Page one <ref:2605.10458#pg1>.
Lalam: I’m pretty sure this work establishes a new standard for how we should approach out-of-distribution evaluation of AI models for QTA properties > Page one <ref:2605.10458#pg1>.
Tom: It’s solid work that provides concrete methods and sets a new benchmark for measuring success in this area > Page one <ref:2605.10458#pg1>.
Jane: And it shows the field is moving toward focusing more on data quality and representation design rather than just architectural tricks > Page one <ref:2605.10458#pg1>.
Title and authors: Tom: Now, what about the specific improvements they suggest to existing methods? Where do they find gains in performance?
Jane: They suggest focusing on dense connectivity for scalar models when you use data augmentation, and they found that comparing E(three)-equivariant and non-equivariant models shows that scalar models benefit a lot from those dense graphs when augmentation is introduced > Page six <ref:2605.10458#pg6>.
Lu: But they also noted that without any data augmentation, the difference between those two model types isn't significant if you keep connectivity the same, meaning the depth is being invested more into learning geometry > Page six <ref:2605.10458#pg6>.
Meng: And on the other side, they found that models using ground truth QTA properties as input features give a statistically significant boost for properties like alpha, U zero, and Cv at every training fraction > Page ten <ref:2605.10458#pg10>.
Tom: So it seems the paper is arguing that using these physically consistent properties as input gives a real statistical advantage to other predictions > Page ten <ref:2605.10458#pg10>.
Jane: They are using a repeated measures ANOVA with Tukey’s honestly significant difference framework to show that non-equivariant, rotationally augmented models outperform their equivariant counterparts significantly > Page ten <ref:2605.10458#pg10>.
Lu: That’s the formal way of saying that for these atomic properties, you should probably lean toward the non-equivariant design > Page ten <ref:2605.10458#pg10>.
Tom: What’s the final word on this paper? What are we actually supposed to do with this information now?
Jane: To wrap up "QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space," they’ve shown a new way to evaluate atomic property models that doesn't rely on flawed sampling methods > Page one <ref:2605.10458#pg1>, QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space>.
Lu: They have introduced the QT-Net model and proven it can infer properties for atoms outside their training set and that those inferred properties actually help improve other predictions > Page one <ref:2605.10458#pg1>.
Meng: The main implication is that we need better ways of knowing when an AI model is truly generalizing and not just overfitting on the specific molecular space it was shown > Page one <ref:2605.10458#pg1>.
Tom: It’s solid work that provides concrete methods and sets a new benchmark for how we measure success in this area > Page one <ref:2605.10458#pg1>.
Jane: And it shows the field is moving toward focusing more on data quality and representation design rather than just architectural tricks > Page one <ref:2605.10458#pg1>.
The paper's summary: Tom: So we've gone through how they set up their testing and compared their models, but let’s distill what QT-Net actually *is* in plain English > Page one <ref:2605.10458#pg1>.
Jane: Basically, they built a new way to check if an AI is actually good at predicting atomic properties, like charges or dipoles > Page one <ref:2605.10458#pg1>.
Lu: They say that the old way of checking—just testing random molecules—was flawed because it didn't make sure the test atoms were in totally new chemical environments > Page three <ref:2605.10458#pg3>.
Meng: So they create these atomic environment clusters using something called SOAP descriptors and then they only test on clusters the model has never seen before > Page three <ref:2605.10458#pg3>.
Tom: And the metrics they use only care about those unseen clusters, which they argue gives a much more honest picture of out-of-distribution performance > Page one <ref:2605.10458#pg1>.
Jane: They introduce QT-Net as this new model—a graph neural network that isn't rotationally aware, but it incorporates some rotational information in a specific way > Page one <ref:2605.10458#pg1>.
Lu: This model is trained to learn these quantum topological properties, those electron population things and multipoles like the quadrupole moment > Page two <ref:2605.10458#pg2>.
Tom: And this isn't just theoretical; they show you can use this QT-Net to guess properties for atoms in molecules from QM9 that were totally outside of their training data > Page one <ref:2605.10458#pg1>.
Jane: That’s the part that really interests me, because they then test if those inferred properties actually help when you use them as inputs for other property predictions > Page one <ref:2605.10458#pg1>.
Lu: They validate this by showing that the dipole moments calculated from QT-Net's outputs match the real, ground truth values they got from QM9 > Page one <ref:2605.10458#pg1>.
Tom: That’s a pretty solid check, proving that what the model learns physically makes sense on a real molecule > Page one <ref:2605.10458#pg1>.
Jane: So the big picture here is that they are shifting the focus away from just making models bigger or more complex > Page one <ref:2605.10458#pg1>.
Lu: The paper suggests the real bottleneck isn't how to build a better model for these targets > Page one <ref:2605.10458#pg1>.
Tom: It’s more about which atomic properties you even bother trying to model and on what kind of chemistry that matters most > Page one <ref:2605.10458#pg1>.
Jane: And for anyone working in production, it means we need a better way to know when an AI is actually generalizing and not just memorizing the specific set of molecules it was shown > Page one <ref:2605.10458#pg1>.
Lu: It’s establishing a new standard for how we evaluate these atomic property models, focusing heavily on data quality and representation design instead of just architectural tricks > Page one <ref:2605.10458#pg1>.
Tom: This is a solid piece of work that gives us concrete methods to measure success in this area > Page one <ref:2605.10458#pg1>.
Jane: And it shows the field is moving toward focusing more on data quality and representation design rather than just architectural tricks > Page one <ref:2605.10458#pg1>.
The paper's improvements: Tom: We’ve covered how they set up their testing and compared their models, but now let’s look at what improvements they suggest for the underlying methods themselves > Page six <ref:2605.10458#pg6>.
Jane: They are suggesting a few ways to make the existing evaluation methods better, specifically around how connectivity works when you use data augmentation > Page six <ref:2605.10458#pg6>.
Lu: They found that for scalar models, if you introduce data augmentation and keep the connectivity the same, those models benefit a lot from being densely connected graphs > Page six <ref:2605.10458#pg6>.
Tom: So it sounds like they think more connections help capture the geometry when you're adding extra training data > Page six <ref:2605.10458#pg6>.
Jane: But they also noted that without any augmentation, if you keep the connectivity equal, you won't see a big difference between the equivariant and non-equivariant models > Page six <ref:2605.10458#pg6>.
Lu: That means when there's no extra data, the model is actually spending its training time focusing more on learning the actual geometry of things > Page six <ref:2605.10458#pg6>.
Tom: Okay, so if you want to use these models for something practical, they recommend leaning toward that non-equivariant design since it performed better statistically on the comparison > Page ten <ref:2605.10458#pg10>.
Jane: And they also have this framework using repeated measures ANOVA with Tukey’s honest significant difference to prove those non-equivariant, rotationally augmented models actually beat their equivariant versions significantly > Page ten <ref:2605.10458#pg10>.
Lu: That’s the formal proof that for these atomic properties, you should probably lean toward the non-equivariant design when you're building your system > Page ten <ref:2605.10458#pg10>.
Tom: And they also found that using those ground truth QTA properties as input features gives a statistically significant boost for things like alpha, U zero, and Cv at every training fraction > Page ten <ref:2605.10458#pg10>.
Jane: That means if you can get those physical property values into your AI as an input feature, it actually helps the final prediction work better > Page ten <ref:2605.10458#pg10>.
Lu: That’s a strong result because it links the representation learning directly to tangible gains in prediction accuracy > Page ten <ref:2605.10458#pg10>.
Tom: So the main thing is that this paper is telling us that the path forward isn't just about building bigger neural networks > Page ten <ref:2605.10458#pg10>.
Jane: It’s more about figuring out which atomic properties you choose to model and on what chemistry those properties matter most > Page one <ref:2605.10458#pg1>.
Conclusion: Tom: So we’ve covered how they set up their testing and compared their models, but now let’s look at what improvements they suggest for the underlying methods themselves > Page six <ref:2605.10458#pg6>.
Jane: They are suggesting a few ways to make the existing evaluation methods better, specifically around how connectivity works when you use data augmentation > Page six <ref:2605.10458#pg6>.
Lu: They found that for scalar models, if you introduce data augmentation and keep the connectivity the same, those models benefit a lot from being densely connected graphs > Page six <ref:2605.10458#pg6>.
Tom: So it sounds like they think more connections help capture the geometry when you're adding extra training data > Page six <ref:2605.10458#pg6>.
Jane: But they also noted that without any augmentation, if you keep the connectivity equal, you won't see a big difference between the equivariant and non-equivariant models > Page six <ref:2605.10458#pg6>.
Lu: That means when there's no extra data, the model is actually spending its training time focusing more on learning the actual geometry of things > Page six <ref:2605.10458#pg6>.
Tom: Okay, so if you want to use these models for something practical, they recommend leaning toward that non-equivariant design since it performed better statistically on the comparison > Page ten <ref:2605.10458#pg10>.
Jane: And they also have this framework using repeated measures ANOVA with Tukey’s honest significant difference to prove those non-equivariant, rotationally augmented models actually beat their equivariant versions significantly > Page ten <ref:2605.10458#pg10>.
Lu: That’s the formal way of saying that for these atomic properties, you should probably lean toward the non-equivariant design when you're building your system > Page ten <ref:2605.10458#pg10>.
Tom: And they also found that using those ground truth QTA properties as input features gives a statistically significant boost for things like alpha, U zero, and Cv at every training fraction > Page ten <ref:2605.10458#pg10>.
Jane: That means if you can get those physical property values into your AI as an input feature, it actually helps the final prediction work better > Page ten <ref:2605.10458#pg10>.
Lu: That’s a strong result because it links the representation learning directly to tangible gains in prediction accuracy > Page ten <ref:2605.10458#pg10>.
Tom: So the main thing is that this paper is telling us that the path forward isn't just about building bigger neural networks > Page ten <ref:2605.10458#pg10>.
Jane: It’s more about figuring out which atomic properties you choose to model and on what chemistry those properties matter most > Page one <ref:2605.10458#pg1>.
Tom: To wrap up on QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space, they’ve shown a new way to evaluate atomic property models that doesn't rely on flawed sampling methods > Page one <ref:2605.10458#pg1>.
Jane: They have introduced the QT-Net model and proven it can infer properties for atoms outside their training set and that those inferred properties actually help improve other predictions > Page one <ref:2605.10458#pg1>.
Lu: The main implication is that the bottleneck has shifted from how to model these targets to which targets you choose to model and on what chemistry matters most > Page one <ref:2605.10458#pg1>.
Meng: From a production standpoint, it means we need better ways to know when an AI is truly generalizing and not just overfitting on the specific molecular space it was shown > Page one <ref:2605.10458#pg1>.
Lalam: I’m pretty sure this work establishes a new standard for how we should approach out-of-distribution evaluation of AI models for QTA properties > Page one <ref:2605.10458#pg1>.
Tom: It’s a solid piece of work that provides concrete methods, and it sets a new benchmark for how we measure success in this area > Page one <ref:2605.10458#pg1>.
Jane: And it shows that the field is moving toward focusing more on data quality and representation design rather than just architectural tricks > Page one <ref:2605.10458#pg1>.
Pablo Martínez Crespo, Stefano Ribes, Martin Rahm, Richard Beckmann, Robert S. Jordan Technology Research Intel Corporation Marisa Gliege Chief Technology Office EMD Electronics Santiago Miret Lila Sciences Vijay Kris Narasimhan Chief Technology Office M Ventures Rocó Mercado Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg
Chalmers University of Technology · University of Gothenburg · Intel Corporation · EMD Electronics · Lila Sciences · M Ventures
cs.LG, cond-mat.mtrl-sci, physics.chem-ph
Submitted: 2026-05-11
Updated: 2026-11-09
Code: https://github.com/pablomcrespo/QT-Net
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 84/100
The gist: Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property prediction…
Key concepts
- SOAP Descriptors
- These are mathematical representations used to describe the local chemical environment surrounding an atom. They capture the spatial arrangement and atomic species nearby, allowing the model to learn from structural context rather than just individual atom features. They are crucial for clustering similar atomic neighborhoods.
- E(3)-Equivariant Models
- These are neural network architectures designed to respect the physical symmetries of molecules, specifically rotations in 3D space. Equivariance means that if you rotate the input molecule, the model's output (like predicted properties) rotates in a predictable way. This ensures physically consistent predictions.
- Quantum Topological Neural Network (QT-Net)
- This is a non-equivariant graph neural network designed to learn Quantum Topological Atomic Information Moments (QTAIM). It is rotationally augmented, meaning it incorporates rotational information into its learning process. QT-Net can infer atomic properties for molecules outside the training data and validates its findings by recovering known molecular dipole moments.
- Held-out Evaluation Protocol
- This novel method ensures a fair test of a model's generalization ability to unseen chemical environments. It involves clustering atoms into environment groups using SOAP descriptors and then only testing the model on cluster labels that were never seen during training, providing a faithful assessment of OOD performance.
Terminology
Summary
Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property prediction, but their evaluation as machine learning targets has been complicated by the absence of a principled out-of-distribution evaluation protocol at the atomic level. p. 0 mercado@chalmers. se
Abstract
Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property prediction, but
their evaluation as machine learning targets has been complicated by the absence
of a principled out-of-distribution evaluation protocol at the atomic level. se
Abstract
Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property prediction, but
their evaluation as machine learning targets has been complicated by the absence
of a principled out-of-distribution evaluation protocol at the atomic level. In this
work, we propose a held-out evaluation protocol that clusters atomic environments
by SOAP descriptors and computes metrics accounting only for cluster labels
unseen during training. In this
work, we propose a held-out evaluation protocol that clusters atomic environments
by SOAP descriptors and computes metrics accounting only for cluster labels
unseen during training. Following this procedure, we use 5×5 cross-validation
and Tukey’s HSD to run a statistically rigorous comparison of E(3)-equivariant
against non-equivariant, rotationally augmented models for predicting electron
populations and multipoles of H, C, N, and O atoms. Following this procedure, we use 5×5 cross-validation
and Tukey’s HSD to run a statistically rigorous comparison of E(3)-equivariant
against non-equivariant, rotationally augmented models for predicting electron
populations and multipoles of H, C, N, and O atoms. Building on our results,
we introduce the Quantum Topological Neural Network (QT-Net), a rotationally
augmented, non-equivariant graph neural network. Building on our results,
we introduce the Quantum Topological Neural Network (QT-Net), a rotationally
augmented, non-equivariant graph neural network. We show that QT-Net can be
used to infer properties of atoms in molecules from QM9 outside our training
set, and that these inferred properties can yield improvement when used as input
features for downstream molecular property prediction. We show that QT-Net can be
used to infer properties of atoms in molecules from QM9 outside our training
set, and that these inferred properties can yield improvement when used as input
features for downstream molecular property prediction. To further validate the
framework, molecular dipole moments computed from QT-Net’s per-atom outputs
recover the ground-truth values reported in QM9.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The paper addresses the challenge of out-of-distribution (OOD) generalization and broad sampling in molecular property prediction by proposing a novel evaluation protocol for atomic properties. The authors argue that metrics that account for all atoms in a test set of molecules, rather than only atoms in environments unseen during training, lead to overconfident results that don’t reflect true OOD performance. p. 1 Atoms, as the building
blocks of chemistry and molecules, are open quantum systems with complex interactions of their
own dictated by their electronic structure. In this work, we show that metrics that account for all
atoms in a test set of molecules, rather than only atoms in environments unseen during training, lead
to overconfident results that don’t reflect true OOD performance. Building on our results,
we introduce the Quantum Topological Neural Network (QT-Net), a rotationally
augmented, non-equivariant graph neural network. We show that QT-Net can be
used to infer properties of atoms in molecules from QM9 outside our training
set, and that these inferred properties can yield improvement when used as input
features for downstream molecular property prediction.Where the paper says it
Where the paper says it
How the training signal is built
The dataset construction involves grouping atoms into element-specific clusters of similar local environment using SOAP descriptors and then clustering these environments by atomic species using HDBSCAN to create held-out test sets of atoms whose cluster labels never appear in the training data. For the molecular property prediction experiments, they included QM9 molecules outside AIMEl, where QTAIM properties were inferred using the trained models. p. 3 Color for the other atoms: H , C , N , O . For the molecular property prediction experiments, we included the QM9 molecules outside AIMEl,
excluding those containing fluorine (as there was no data for it in AIMEl), yielding 98 096 molecules
whose QTAIM properties were inferred using the trained models. Atomic properties depend on the shape
of this basin, as they are quantities integrated over it. In this work, we focus on electron population
(N), atomic contribution to molecular dipole moment (µ), traceless quadrupole moment (Q), and
localization index (λ),
N =
Z
Ω
ρ(r)dr, µ = −
Z
Ω
e r
(Ω)ρ(r)dr +
N
Xb(Ω)
Λ
[RΩ − Rb(Ω|Λ)] q(Ω|Λ),
and Qαβ = −
Z
Ω
e ρ(r)
3r
(Ω)
α r
(Ω)
β − δαβ∥r
(Ω)∥
2
dr.Where the paper says it
Where the paper says it
Where the gains come from
The comparison between E(3)-equivariant and non-equivariant models suggests that scalar models benefit from densely connected graphs for capturing geometrical information, especially when data augmentation is used. The results in Table 9 show that the difference between EGNN and SGNN is not significant at equal connectivity and without data augmentation, indicating that the greater depth is being invested in geometric learning. p. 6 The results in Table 2 show that non-equivariant, rotationally augmented models outperform the
equivariant counterparts significantly. Looking at Table 1, one would be tempted to attribute this
advantage to the greater depth of the scalar models. 2 Quality of the inferred QTA properties
Our results for molecular property prediction on the AIMEl dataset are found in Section A. Therein, we show that the performance gain from ground truth QTA properties-informed models
is statistically significant based on Table 10, and show the R2
learning curves in Figure 11. As a matter of fact, the observed outliers suggest that QM9 contains atoms in regions of atomic
chemical space far from those in AIMEl. This recovery is a strong physical consistency check:
because molecular dipole moments arise from the vector sum of per-atom contributions, accurate
reconstruction requires that QT-Net has correctly learned not only the magnitudes but also the
8Where the paper says it
Where the paper says it
Where the paper says it
Evaluation and Conclusion
The authors use a repeated-measures ANOVA (RM-ANOVA) with Tukey’s honestly significant difference (HSD) framework to compare model performance, finding that non-equivariant, rotationally augmented models outperform their equivariant counterparts significantly. The final conclusion suggests that OOD evaluation of AI models for QTA properties requires careful labeling and tracking of the atomic environments. p. 9 can reconstruct molecular dipole moments outside the set of training molecules from inferred atomic
contributions. We want to emphasize that OOD evaluation of AI models for QTA properties requires careful labeling
and tracking of the atomic environments. Our results suggest that further progress on QTA property prediction is now primarily a data problem
rather than a representation or architectural challenge: across the four properties studied here, the
same architecture achieves CCC values ranging from ∼0. 99 depending on element and
property, indicating that the bottleneck has shifted from how to model these targets to which targets
to model and on what chemistry. This is a pitfall that has been repeatedly observed in the AI-for-QCT literature (see Section 2). Our results suggest that further progress on QTA property prediction is now primarily a data problem
rather than a representation or architectural challenge: across the four properties studied here, the
same architecture achieves CCC values ranging from ∼0.Where the paper says it
Where the paper says it
Where the paper says it
Code and data availability
Code and data for this project are available at https://github.com/pablomcrespo/QT-Net, including all seeds and information to reproduce the experiments of this work. p. 9 com/pablomcrespo/QT-Net. It
includes all seeds and information to reproduce the experiments of this work, besides relevant code to
reproduce the statistical analysis. The source data is referenced in the text, but we include compressed
versions of our curated files (mainly for reproducibility with the same cluster labels). The QTAIM
properties inferred by QT-Net are released in the repository, too.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Statistically rigorous evaluation protocol for atomic properties: The proposed held-out evaluation protocol clusters atomic environments by SOAP descriptors and computes metrics accounting only for cluster labels unseen during training, which leads to
faithful OOD evaluation.
-
Model comparison framework: Use repeated means ANOVA with Tukey’s honestly significant difference (HSD) frameworks to compare E(3)-equivariant against non-equivariant models for predicting scalar and tensor properties, allowing for
statistically significant benchmarking.
-
Inferential quality assessment: The QT-Net can be used to
infer properties of atoms in molecules from QM9 outside our training set,
and these inferred properties can yield improvement when used as input features for downstream molecular property prediction. -
Property recovery validation: Molecular dipole moments computed from QT-Net’s per-atom outputs
recover the ground-truth values reported in QM9,
validating the physical consistency of the learned QTA properties. -
Architectural selection: Based on results showing that
non-equivariant, rotationally augmented models outperform the equivariant counterparts significantly,
select a non-equivariant design for QT-Net to reconstruct molecular dipole moments outside training sets from inferred atomic contributions.
Abstract
Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property prediction, but their evaluation as machine learning targets has been complicated by the absence of a principled out-of-distribution evaluation protocol at the atomic level. In this work, we propose a held-out evaluation protocol that clusters atomic environments by SOAP descriptors and computes metrics accounting only for cluster labels unseen during training. Following this procedure, we use 5 times 5 cross-validation and Tukey's HSD to run a statistically rigorous comparison of E(3)-equivariant against non-equivariant, rotationally augmented models for predicting electron populations and multipoles of H, C, N, and O atoms. Building on our results, we introduce the Quantum Topological Neural Network (QT-Net), a rotationally augmented, non-equivariant graph neural network. We show that QT-Net can be used to infer properties of atoms in molecules from QM9 outside our training set, and that the atomic environments seen during training strongly affect the performance of molecular property prediction tasks. To further validate the framework, molecular dipole moments computed from QT-Net's per-atom outputs recover the ground-truth values reported in QM9. We release all code and data, including a JAX implementation of QT-Net.
Sources
- BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
- MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields
- Directional Message Passing for Molecular Graphs
- SemlaFlow -- Efficient 3D Molecular Generation with Latent Attention and Equivariant Flow Matching
- The Open Molecules 2025 (OMol25) Dataset, Evaluations, and Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis