Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance
cs.AI, cs.CL
Submitted: 2026-02-26
Updated: 2026-10-12
Comments: Accepted at ICML 2026
Code: https://github.com/lwd17/strategy-execute-pipeline
License: http://creativecommons.org/licenses/by/4.0/
The gist: Example-based guidance is widely used to improve mathematical reasoning at inference time, yet its effectiveness is highly unstable across problems and models-even when the guidance is correct and
Terminology
Abstract
Example-based guidance is widely used to improve mathematical reasoning at inference time, yet its effectiveness is highly unstable across problems and models-even when the guidance is correct and problem-relevant. We show that this instability arises from a previously underexplored gap between strategy usage-whether a reasoning strategy appears in successful solutions-and strategy executability-whether the strategy remains effective when instantiated as guidance for a target model. Through a controlled analysis of paired human-written and model-generated solutions, we identify a systematic dissociation between usage and executability: human- and model-derived strategies differ in structured, domain-dependent ways, leading to complementary strengths and consistent source-dependent reversals under guidance. Building on this diagnosis, we propose Selective Strategy Retrieval (SSR), a test-time framework that explicitly models executability by selectively retrieving and combining strategies using empirical, multi-route, source-aware signals. Across multiple mathematical reasoning benchmarks, SSR yields reliable and consistent improvements over direct solving, in-context learning, and single-source guidance, improving accuracy by up to +13 points on AIME25 and +5 points on Apex for compact reasoning models. Code and benchmark are publicly available at: https://github.com/lwd17/strategy-execute-pipeline.
Sources
- GPT-4 Technical Report
- Zero-Shot Verification-guided Chain of Thoughts
- Training Verifiers to Solve Math Word Problems
- Faithful Reasoning Using Large Language Models
- Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
- Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Coarse-to-Fine Process Reward Modeling for Mathematical Reasoning
- Are self-explanations from Large Language Models faithful?
- Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematics
- Premise-Augmented Reasoning Chains Improve Error Identification in Math reasoning with LLMs
- Self-Training Elicits Concise Reasoning in Large Language Models
- Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs
- Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
- Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
- Accurate and Diverse LLM Mathematical Reasoning via Automated PRM-Guided GFlowNets
- HARP: A challenging human-annotated math reasoning benchmark
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems