Fusion Training for Mathematical Generalization in Large Language Models

This paper presents a systematic study of Thinking Mode Fusion (TMF), a technique that enables large language models to support both concise responses and long-form reasoning within a single model. The authors investigate how training dynamics—specifically the data ratio and training schedule between thinking and non-thinking modes—affect model performance in mathematical problem solving.

The study constructs a benchmark called Fusion Bench using GSM8K for non-thinking mode data and OpenMathReasoning for thinking mode data. They fix the thinking mode training data at 1,500 examples and vary the non-thinking data size to achieve ratios from 1:4 to 4:1. Three training schedules are examined: T-NT (thinking before non-thinking), NT-T (non-thinking before thinking), and Mix (interleaved training).

The key findings reveal an asymmetric interaction between the two modes. The paper states: increasing the ratio of non-thinking supervision reduces the accuracy of the thinking mode. When averaging results across all data ratios, the Mix training schedule achieves the highest accuracy on both test sets, with accuracies of 0.208 for thinking and 0.695 for non-thinking modes.

The results show that training schedules significantly modulate the trade-off between modes. At a fixed 1:1 data ratio, the Mix schedule achieves the highest non-thinking accuracy, while the NT-T schedule achieves the highest thinking accuracy. The T-NT schedule shows degradation in non-thinking performance compared to the base model baseline.

Data ratio analysis reveals that the accuracy of the thinking mode exhibits a consistent increase as the data ratio increases across all training schedules, while non-thinking accuracy shows varying trends depending on the schedule. The optimal training schedule is strongly coupled with the data ratio, with T-NT preferable when non-thinking data dominate and Mix preferable when thinking data dominate.

The paper quantifies a negative correlation between the two modes: "non-thinking and thinking accuracies exhibit a negative correlation (r = −0.58), with a coefficient of determination of R2 = 0.34. The estimated regression slope (−0.91) indicates that a 0.01 increase in non-thinking mode performance is associated with an average decrease of approximately 0.0091 in thinking mode performance."

The authors conclude that the training schedule modulates this trade-off: when averaging results across all data ratios, the Mix training schedule achieves the highest accuracy on both the thinking and non-thinking test sets. They also note that the optimal training schedule depends on the data ratio, and that the Mix training schedule performs well across varying data ratios for both modes.

The study's limitations include restriction to mathematical reasoning, fixed thinking data amount, and the open challenge of developing training strategies to explicitly mitigate the tension between modes. All code and data are released at Fusion Bench.

Improvements to AI Systems:

1. Adaptive Mode-Switching via Data-Ratio-Aware Scheduling: Implement a training scheduler that dynamically adjusts the ratio of thinking to non-thinking data based on the target deployment profile. For systems prioritizing concise, high-volume responses (e.g., customer support), use a T-NT schedule with a 4:1 non-thinking-to-thinking ratio to maximize non-thinking accuracy. For systems prioritizing deep reasoning (e.g., theorem proving), use an NT-T schedule with a 4:1 thinking-to-non-thinking ratio. For balanced general-purpose assistants, use the Mix schedule at a 1:1 ratio, which yields the best average performance (0.208 thinking, 0.695 non-thinking accuracy).

2. Trade-off Quantification and Calibration: Integrate the measured negative correlation (r = −0.58, slope = −0.91) into a calibration module. When a user request is classified as quick factual (non-thinking), the system can proactively reduce its internal reasoning depth to save compute, knowing that a 0.01 gain in non-thinking accuracy costs 0.0091 in thinking accuracy. Conversely, for complex math or logic queries, the system can allocate more reasoning tokens, accepting a predictable drop in speed for higher thinking-mode accuracy.

3. Schedule-Based Curriculum Learning for Multi-Task Models: Use the finding that Mix schedule outperforms both T-NT and NT-T on average to design a default training curriculum for new LLMs. Instead of sequential fine-tuning, interleave thinking and non-thinking examples in a shuffled manner. This reduces catastrophic forgetting and maintains both response modes without requiring separate model checkpoints, enabling a single model to serve both API endpoints (fast and reasoning) with one set of weights.

4. Data Ratio Optimization for Resource-Constrained Training: Given that thinking-mode accuracy consistently increases with data ratio across all schedules, allocate additional training data to thinking mode when compute is limited but reasoning quality is critical. The paper’s fixed 1,500 thinking examples suggests that even small additions of high-quality reasoning data (e.g., OpenMathReasoning) can yield measurable gains, so prioritize sourcing diverse thinking-mode datasets over scaling non-thinking data.

5. Predictive Mode Selection for User Queries: Build a lightweight classifier that estimates whether a query benefits from thinking mode (e.g., multi-step math, logical deduction) or non-thinking mode (e.g., simple facts, greetings). Train this classifier using the observed trade-off: when the classifier predicts high thinking-mode benefit, route to the model’s reasoning path; otherwise, use the concise path. This leverages the Mix schedule’s balanced performance to serve both user types without latency penalties.

What the Improved AI System Can Do:

• Serve dual-response modes from a single model with predictable accuracy trade-offs, eliminating the need for separate fast and reasoning models.

• Dynamically adjust its reasoning depth based on query complexity and user context, optimizing for speed or accuracy as needed.

• Achieve up to 0.695 non-thinking accuracy on math word problems while retaining 0.208 thinking accuracy on reasoning benchmarks, with the ability to shift this balance by ±10% based on deployment needs.

• Avoid performance collapse in non-thinking mode (which occurs with T-NT schedules) by defaulting to Mix training, ensuring robust baseline performance across all query types.

• Provide a quantitative guarantee that increasing non-thinking performance by 1% will cost ~0.9% in thinking performance, enabling system designers to make informed trade-offs for their specific application.

Thinking Mode Fusion (TMF) enables large language models to support both concise responses and long-form reasoning by unifying a non-thinking mode and a thinking mode within a single model. However, its training dynamics, including the data ratio and training schedule between the two modes, remain underexplored. In this work, we present a systematic study of TMF by analyzing the effects of the training schedule and data ratio between thinking and non-thinking modes. Focusing on mathematical problem solving, we construct a benchmark with multiple thinking-to-non-thinking data ratios and three training schedules. Our results reveal an asymmetric interaction between the two modes: increasing the ratio of non-thinking supervision reduces the accuracy of the thinking mode. We further show that different training schedules modulate this trade-off and that the optimal schedule depends on the data ratio. Finally, we quantify a negative correlation between non-thinking and thinking mode supervision, highlighting an inherent tension between these two modes. These findings provide practical guidance for designing effective TMF training settings. All code and data are released to support further research at: Fusion Bench.