gwBenchmarks: Stress-Testing LLM Agents on High-Precision Gravitational Wave Astronomy

arXiv:2605.11269 · gr-qc, astro-ph.HE, astro-ph.IM, cs.AI · Submitted 2026-05-11 · Read on arXiv

gr-qc, astro-ph.HE, astro-ph.IM, cs.AI

Submitted: 2026-05-11

Updated: 2026-09-20

Comments: 26 pages, 4 figures

Code: https://github.com/tousifislam/gwBenchmarks

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 109× in this paper · explained in 2 episodes
• Accuracy — 29× in this paper · explained in 1 episode
• Failure Modes — 6× in this paper · explained in 3 episodes
• Verification — 6× in this paper · explained in 2 episodes
• Benchmarking — 5× in this paper · explained in 3 episodes
• Fidelity — 5× in this paper · explained in 2 episodes
• Foundation Model — 5× in this paper · explained in 3 episodes
• Foundation Models — 5× in this paper · explained in 7 episodes
• Core — 4× in this paper · explained in 2 episodes
• General Relativity — 3× in this paper
• Generalization — 3× in this paper · explained in 13 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Scaling Laws — 2× in this paper · explained in 4 episodes
• Test-time Scaling — 2× in this paper · explained in 2 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Agentic Workflows — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Chebyshev Polynomials — 1× in this paper · explained in 2 episodes
• Domain Adaptation — 1× in this paper · explained in 5 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Redundancy — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Scaffolding — 1× in this paper · explained in 3 episodes
• Singular Value Decomposition (SVD) — 1× in this paper · explained in 2 episodes
• Standardized Metrics — 1× in this paper · explained in 3 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• GAIA: a benchmark for General AI Assistants
• ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
• Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
• Program Synthesis with Large Language Models
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• Evaluating Large Language Models Trained on Code
• Training Verifiers to Solve Math Word Problems
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
• Reflexion: Language Agents with Verbal Reinforcement Learning
• The Denario project: Deep knowledge AI agents for scientific discovery
• CosmoBench: A Multiscale, Multiview, Multitask Cosmology Benchmark for Geometric Deep Learning
• Accurate models for recoil velocity distribution in black hole mergers with comparable to extreme mass-ratios and their astrophysical implications
• Kick matters: The impact of a new recoil model on the retention of hierarchical black-hole remnants in globular clusters
• New approach to template banks of gravitational waves with higher harmonics: Reducing matched-filtering cost by over an order of magnitude
• Generating optimal Gravitational-Wave template banks with metric-preserving autoencoders
• New black hole mergers in the LIGO-Virgo O3 data from a gravitational wave search including higher-order harmonics
• Searching for precessing binary systems with mode-by-mode filtering and marginalization

Related papers

• Listening to the Horizon: Probing Near-Horizon Reflectivity with GW250114
• A Pairwise Surrogate for Gravitational-Wave Spectra from Highly Relativistic Vacuum Bubble Collisions
• Tests of General Relativity with Einstein Telescope
• A weakly modelled view of the joint compact-binary mass plane: population structure and spectral-siren cosmology
• Unitary quantum matter-bounce in a universe with a positive cosmological constant
• Nonreciprocal black holes with primary hair in beyond Horndeski gravity: Exact solutions and stability