MOOD: Benchmarking Post-Hoc OOD Detection for Materials Property Prediction
cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Comments: Accepted to NeurIPS 2026 (Evaluations & Datasets Track)
Code: https://github.com/vertaix/mood-benchmark
License: http://creativecommons.org/licenses/by/4.0/
The gist: Out-of-distribution (OOD) detection is essential for deploying machine learning models in high stakes scientific applications…
Terminology
Abstract
Out-of-distribution (OOD) detection is essential for deploying machine learning models in high stakes scientific applications, yet methods that flag OOD inputs without retraining the model have been validated almost exclusively on image classification. We introduce MOOD (Materials Out-of-Distribution Detection), a benchmark for evaluating whether these post-hoc OOD detectors transfer to graph neural networks (GNNs) on materials property prediction. MOOD evaluates 15 post-hoc detectors and 2 supervised diagnostic probes across 4 GNN architectures, 5 MatBench tasks, and 15 physically motivated compositional and structural shifts, yielding 264 evaluation settings and over 13,000 AUROC measurements across three encoder training seeds. Our evaluation reveals three key findings. First, current post-hoc detectors perform near chance; all 15 detectors achieve an AUROC around 0.5. Second, detector failures decompose into two regimes: method bottlenecks and encoder bottlenecks. Method bottlenecks arise when supervised probes can separate in-distribution (ID) from OOD samples but unsupervised detectors fail, meaning that the embeddings contain OOD signal that current detectors do not extract. Encoder bottlenecks arise when even supervised probes fail to separate ID from OOD samples, indicating that the relevant shift is not accessible in the frozen GNN representation. Third, when OOD prediction error exceeds ID error, the AUROC of distance-based and density-based detectors correlates with the size of that gap. Collectively, these findings indicate that reliable OOD detection in scientific domains requires representation learning strategies that explicitly preserve OOD relevant signal, particularly for tasks where current GNN architectures encode no extractable signal at all.
Sources
- BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
- Extremely Simple Activation Shaping for Out-of-Distribution Detection
- Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules
- GOOD: A Graph Out-of-Distribution Benchmark
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Deep Anomaly Detection with Outlier Exposure
- Open Graph Benchmark: Datasets for Machine Learning on Graphs
- A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification
- Energy-based Out-of-distribution Detection
- Structure-based out-of-distribution (OOD) materials property prediction: a benchmark study
- SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
- ReAct: Out-of-distribution Detection With Rectified Activations
- CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
- Contrastive Training for Improved Out-of-Distribution Detection
- OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
- GeSS: Benchmarking Geometric Deep Learning under Scientific Applications with Distribution Shifts
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis