ResearchMath-14K: Scaling Research-Level Mathematics via Agents
cs.CL
Submitted: 2026-05-27
Updated: 2026-09-27
Comments: Work in progress. Dataset available at: https://huggingface.co/datasets/amphora/ResearchMath-14k
Code: https://github.com/huggingface/Math-Verify
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: The frontier of mathematics is defined by problems whose solutions are not yet known.
Terminology
Abstract
The frontier of mathematics is defined by problems whose solutions are not yet known. However, whether language models can meaningfully engage with such problems without human intervention remains unclear. A major obstacle is the lack of large-scale research-level math datasets. To this end, we introduce ResearchMath-14k, a set of 14, 056 problems curated from academic sources via a multi-agent pipeline. ResearchMath-14k spans 11 mathematical domains and ranks above existing math datasets on knowledge, novelty, and procedural difficulty. To our knowledge, it is the largest research-level mathematical problem set available for training. We additionally generate 220 K teacher trajectories through targeted prompting, followed by behavioral filtering. Notably, however, generating correct trajectories is nontrivial at this level, and two LLM judges label only 3.7% and 4.3% of sampled ResearchMath training trajectories as correct. Nevertheless, across three model families, full-parameter training on ResearchMath improves performance on graduate- and research-level mathematics benchmarks by 2.1 points over the starting checkpoints. In comparison, training on existing datasets such as DASD and Nemotron-SFT-Math-v4 changes performance by 0.0 and-0.5 points, respectively. Notably, mixing DASD with ResearchMath yields higher scores than token-matched DASD alone on benchmarks covering olympiad short-form (+2.0), graduate- and research-level short-form (+0.8), graduate- and research-level symbolic (+2.6), and proof evaluation (+5.7). Further analysis suggests that research-level mathematical content and greater reasoning diversity may help explain why ResearchMath provides complementary supervision to contemporary datasets. We make ResearchMath-14k publicly available for future works on research-level mathematical reasoning.
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
- Short proofs in combinatorics and number theory
- Short proofs in combinatorics, probability and number theory II
- Training Verifiers to Solve Math Word Problems
- Agentic Reinforced Policy Optimization
- MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning
- Riemann-Bench: A Benchmark for Moonshot Mathematics
- FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
- Measuring Mathematical Problem Solving With the MATH Dataset
- LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent
- WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents
- Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation
- Humanity's Last Exam
- Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs
- Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
- Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math
- Kimi K2.5: Visual Agentic Intelligence
- Kimi K2: Open Agentic Intelligence
- Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering