LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Models
cs.LG
Submitted: 2025-12-04
Updated: 2026-11-09
Comments: 50 pages, 20 figures, 16 tables
Code: https://github.com/LeMaterial/lemat-genbench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generative machine learning (ML) models hold great promise for accelerating materials discovery through the inverse design of inorganic crystals…
Terminology
Abstract
Generative machine learning (ML) models hold great promise for accelerating materials discovery through the inverse design of inorganic crystals, enabling an unprecedented exploration of chemical space. Yet, the lack of standardized evaluation frameworks makes it challenging to evaluate, compare, and further develop these ML models meaningfully. In this work, we introduce LeMat-GenBench, a unified benchmark for generative models of crystalline materials, supported by a set of evaluation metrics designed to better inform model development and downstream applications. We release both an open-source evaluation suite and a public leaderboard on Hugging Face, and benchmark 12 recent generative models. Results reveal that an increase in stability is accompanied by lower novelty and diversity on average, with no model excelling across all dimensions. Altogether, LeMat-GenBench establishes a reproducible and extensible foundation for fair model comparison and aims to guide the development of more reliable, discovery-oriented generative models for crystalline materials.
Sources
- A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems
- A foundation model for atomistic materials chemistry
- Crystal Diffusion Variational Autoencoder for Periodic Material Generation
- Space Group Constrained Crystal Generation
- Space Group Equivariant Crystal Diffusion
- Space Group Conditional Flow Matching
- Space Group Informed Transformer for Crystalline Materials Generation
- Fine-Tuned Language Models Generate Stable Inorganic Materials as Text
- Foundational Large Language Models for Materials Research
- deCIFer: Crystal Structure Prediction from Powder Diffraction Data using Autoregressive Language Models
- Catalyst GFlowNet for electrocatalyst design: A hydrogen evolution reaction case study
- Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations
- LeMat-Bulk: aggregating, and de-duplicating quantum chemistry materials databases
- UMA: A Family of Universal Models for Atoms
- Orb-v3: atomistic simulation at scale
- Crystal-GFN: sampling crystals with desirable properties and constraints
- PLaID++: A Preference Aligned Language Model for Targeted Inorganic Materials Design
- Scalable Diffusion for Materials Generation
- A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery
- Language Modeling Is Compression
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis