When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation

arXiv:2605.20364 · cs.CL · Submitted 2026-05-19 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "When Reasoning Supervision Hurts".

Jane: The gist The study constructs a large TTCW-based dataset for long-form literary review generation and finds that non-reasoning fine-tuning achieves stronger and more stable performance compared to reasoning supervision in…

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We're moving into the second part of our talk about "When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation." This section looks at what the researchers propose to actually fix those instability problems they found when using reasoning supervision.

Jane: They aren't just stopping at finding that non-reasoning fine-tuning is better; they’re laying out specific engineering improvements for how we build these review generation systems from scratch.

Lu: One major suggestion is designing a more sophisticated dataset construction pipeline, which involves performing metric-wise reviewer scoring, filtering the reviewers for quality first, and then synthesizing those comments into complete TTCW review reports for long stories #pg2.

Meng: That sounds like it would make the data generation process much more rigorous before we even start training any models on it, which is something I need to see implemented in production.

Tom: Precisely. And they suggest using specific prompts for each TTCW dimension, like the ones in their table, so all the reviewer models are operating under the exact same rubric during data creation #pg2.

Jane: That should help with inter-metric consistency during data generation, which is a big hurdle when you’re scoring fourteen different creative dimensions and you need them to all be judged by the same standard.

Lu: Then they highlight that we need to engineer systems to explicitly avoid leaking reasoning-style content into the final answer because we saw how prone those models are to do that #pg2.

Meng: That addresses my biggest concern about deployment—preventing the model from rambling when it should just be spitting out scores and not leaking its internal thoughts.

Tom: And finally, they suggest defining the evaluation score differently, using something like seval equals p times half of smae plus half of sbertscore-f1 #pg1 <ref:2605.20364#pg1>.

Jane: That definition is clever because it rewards models only when they are both structurally parseable and semantically close to the human reference outputs, which is a much tighter quality check.

Lu: That way, you get a more robust measure of quality that checks both the format and the meaning simultaneously during evaluation #pg2.

Tom: So what this means for us is that we need to focus on building these multi-step data pipelines and using smarter evaluation metrics rather than just relying on adding reasoning traces to the model itself #pg1 <ref:2605.20364#pg1>.

Jane: It means we’re moving away from thinking that more reasoning input automatically equals better structured output, and toward a more deliberate, multi-stage engineering approach for high quality generation.

Lu: They open up the next area for research by pointing toward investigating whether higher quality reasoning traces or stronger adaptation methods beyond LoRA can actually make reasoning supervision work better in these specific settings #pg1 <ref:2605.20364#pg1>.

The paper's summary: Tom: Now we’re getting to the core of the paper, so let's summarize what they actually found regarding this TTCW-Based Long-Form Literary Review Generation study. It’s about those results we just discussed.

Jane: The central finding is quite direct: for fixed-format rubric generation, adding reasoning supervision doesn't actually help performance and can even make the output less stable.

Lu: They concluded that non-reasoning supervision consistently achieves stronger and more stable performance across both Qwen3 8B and Qwen3 4B models on this task #pg1 <ref:2605.20364#pg1,achieves stronger and more stable performance>.

Meng: So for me, that means if we're shipping a system to generate those reviews, focusing on the non-reasoning fine-tuning path is the safer bet right now before adding complexity.

Tom: Yeah, they showed that reasoning-supervised models are more prone to sequence level failures and leaking reasoning text into the final report #pg1 <ref:2605.20364#pg1>.

Jane: It really highlights that for tasks demanding precise scores and structured outputs, those complex reasoning traces don't automatically pay off in quality or reliability.

Lu: They also mentioned a limitation in their data building process, saying that since they didn't use human annotators to build this massive dataset of two hundred sixty-three thousand nine hundred eleven stories, the supervision signal is entirely model-generated and might carry some bias #pg1 <ref:2605.20364#pg1>.

Meng: That’s a fair caution; we need to remember that if the training data itself has those biases from the generation pipeline, the resulting model might amplify them when it tries to follow strict rules.

Tom: So they are suggesting that for now, if you want better performance on this specific structured task, sticking to non-reasoning fine-tuning is the safer bet #pg1 <ref:2605.20364#pg1>.

Jane: It shows us that we need to focus on building those multi-step data pipelines and using smarter evaluation metrics instead of just relying on adding reasoning traces to the model itself.

Lu: They pointed toward future work investigating whether higher quality reasoning traces or stronger adaptation methods beyond LoRA could make reasoning supervision more effective in these specific settings #pg1 <ref:2605.20364#pg1>.

The paper's improvements: Tom: Let’s talk specifically about the suggested improvements for the TTCW-Based Long-Form Literary Review Generation paper. We’re looking at what they think needs to change in their methodology.

Jane: They detailed several ways to improve the data generation process, focusing on making it more structured and consistent across all fourteen dimensions.

Lu: They detailed a pipeline that performs metric-wise reviewer scoring, reviewer-quality filtering, and comment synthesis to produce complete TTCW review reports for long stories #pg2 <ref:2605.20364#pg1>.

Meng: That sounds like it would make the data generation process much more rigorous before we even start training any models on it, which is something I need to see implemented in production.

Tom: Exactly. And they suggest using specific prompts for each TTCW dimension, like the ones in their table, so all the reviewer models are operating under the exact same rubric during data creation #pg2.

Lu: Then they highlight that we need to engineer systems to explicitly avoid leaking reasoning-style content into the final answer because we saw how prone those models are to do that #pg2.

Meng: That addresses my biggest concern about deployment—preventing the model from rambling when it should just be spitting out scores and not leaking its internal thoughts.

Tom: And they suggest defining the evaluation score differently, using something like seval equals p times half of smae plus half of sbertscore-f1 #pg1 <ref:2605.20364#pg1>.

Lu: That way, you get a more robust measure of quality that checks both the format and the meaning simultaneously during evaluation #pg2.

Jane: It shows us that we need to focus on building those multi-step data pipelines and using smarter evaluation metrics instead of just relying on adding reasoning traces to the model itself.

Lu: They pointed toward future work investigating whether higher quality reasoning traces or stronger adaptation methods beyond LoRA could make reasoning supervision more effective for these specific settings #pg1 <ref:2605.20364#pg1>.

Meng: For us at the startup, that means we should probably focus on optimizing the non-reasoning path first for stability before we worry about adding complex reasoning layers to our generation pipeline #pg2.

Tom: Yeah, so this paper "When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation" gives us a very clear direction on how to handle structured generation when creativity metrics are involved #pg1 <ref:2605.20364#pg1>.

Jane: It makes you think about where else this idea of supervised versus unsupervised guidance might be relevant in complex AI evaluation tasks.

Lu: We can look at how we use evidence-tree rubric supervision, like DEEPRUBRIC, to see if that structured approach works better than trying to inject raw reasoning traces into the text itself #pg2.

Meng: I’m interested to see if those advanced adaptation methods actually solve the stability issues they identified with the lower-level fine-tuning we saw here #pg1 <ref:2605.20364#pg1>.

Conclusion: Tom: So we've covered the whole story of "When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation," and it really settles on one clear direction for structured generation tasks. We're wrapping up now.

Jane: The main point is that for fixed-format review generation, forcing a model to reason actually makes it less stable and hurts performance.

Lu: They concluded that non-reasoning supervision consistently achieves stronger and more stable performance across both Qwen3 8B and Qwen3 4B models on this task.

Meng: So for me, that means if we're shipping a system to generate those reviews, focusing on the non-reasoning fine-tuning path is the safer bet right now.

Tom: Yeah, they showed that reasoning-supervised models are more prone to sequence level failures and leaking reasoning text into the final report #pg1 <ref:2605.20364#pg1>.

Jane: It really highlights that for tasks demanding precise scores and structured outputs, those complex reasoning traces don't automatically pay off in quality or reliability.

Lu: They also noted a limitation in their data building process, saying that since they didn't use human annotators to build this massive dataset of two hundred sixty-three thousand nine hundred eleven stories, the supervision signal is entirely model-generated and might carry some bias #pg1 <ref:2605.20364#pg1>.

Tom: So they are suggesting that for now, if you want better performance on this specific structured task, sticking to non-reasoning fine-tuning is the safer bet #pg1 <ref:2605.20364#pg1>.

Jane: It shows us that we need to focus on building those multi-step data pipelines and using smarter evaluation metrics instead of just relying on adding reasoning traces to the model itself.

Meng: I'm interested to see if those advanced adaptation methods actually solve the stability issues they identified with the lower-level fine-tuning we saw here #pg2 <ref:2605.20364#pg1>.

Tom: Yeah, so for now, it’s a solid piece of evidence showing that for fixed-format tasks, you might want to lean into non-reasoning supervision unless you have a really strong reason to keep adding that reasoning layer #pg1 <ref:2605.20364#pg1>.

Jane: It’s a clear signal for us: when building systems for structured evaluation, focus on data quality and stable fine tuning before trying to force complex reasoning traces in.

Lu: Next time, we might look at how models handle long-term memory, like with ICM-Bench, because that seems connected to the stability issues they found here #pg1 <ref:2605.20364#pg1>.

Jinlong Liu, Mohammed Bahja, Mark Lee

School of Computer Science, University of Birmingham

cs.CL

Submitted: 2026-05-19

Updated: 2026-10-14

Comments: Submit to NAACL 2027

Code: https://github.com/Vince-Liuss/TTCW-based-Review

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 87/100

The gist: The gist The study constructs a large TTCW-based dataset for long-form literary review generation and finds that non-reasoning fine-tuning achieves stronger and more stable performance compared to

Key concepts

Torrance Test of Creative Writing (TTCW): TTCW is a structured framework used to evaluate creativity in writing. It breaks down creative writing into 14 specific dimensions that can be scored, helping researchers systematically assess different aspects of a piece's originality and flexibility.
Reasoning Supervision: This fine-tuning method involves training an AI model using explicit 'reasoning traces'—step-by-step thought processes. The researchers tested if showing the model how to reason helps it generate structured reviews, but found this approach was unstable for fixed formats.
Non-reasoning Fine-tuning: This method involves training the AI model on the review task without providing explicit reasoning steps. The results showed that this approach yielded stronger and more stable performance for generating structured, rubric-based literary reviews across different model sizes.

Terminology

Fine-Tuning — 2× in this paper · explained in 5 episodes
Although the Torrance Test of Creative Writing — 1× in this paper

Summary

The gist The study constructs a large TTCW-based dataset for long-form literary review generation and finds that non-reasoning fine-tuning achieves stronger and more stable performance compared to reasoning supervision in this structured rubric-based review task.

Dataset Construction

The researchers address the lack of a public dataset by constructing 263,911 long-form stories, each annotated with scalar scores and metasynthesised review comments across 14 TTCWbased dimensions The dataset is created by converting original TTCW binary questions into scalar rating questions from 1 to 10 This process involves asking three reviewer models to score all 14 TTCW metrics independently for each story, evaluating reviewer quality through score distribution, discrimination, and metric-isolation analyses The resulting dataset contains long-form stories in the 4K–8K word range, each paired with a complete TTCW-based review report

Reviewer Model Selection and Filtering

Three recent and capable reviewer models were selected for the construction pipeline: Llama-3 3-Nemotron-Super-49B-v1 5, Qwen3-Next-80B-A3B Instruct (non-reasoning mode), and gpt-oss 120b The researchers evaluated reviewer suitability using three diagnostics: score distribution, discrimination score, and metric isolation Qwen3-Next-80B-A3B Instruct was excluded from the synthesis stage because it showed weaker reviewer suitability due to strong score concentration and weak practical discrimination in its performance metrics The final dataset contains 263,911 rows, designed for longcontext literary review generation

Fine-tuning Comparison

The study compares models fine-tuned with and without reasoning content on this structured rubric-based review task Results show that non-reasoning fine-tuning achieves stronger and more stable performance, with the best setting reaching an evaluation score of 0.6820 under the non-reasoning condition Further analysis indicates that reasoning-supervised models are more prone to parse failures, often continuing with irrelevant or repetitive reasoning-style text rather than completing the required 14-metric review report This suggests that for fixed-format rubric-based review generation, reasoning supervision is not straightforwardly beneficial and that precise metric-aligned scoring remains challenging even after task-specific fine-tuning

Evaluation Metrics and Stability

The final evaluation score (Seval) is a combination of parse stability, score quality, and review similarity: Seval = p · (0.5 sMAE + 0.5 sBERTScore-F1) Models fine-tuned without reasoning content are highly reliable across both model scales and decoding modes with high parse rates In contrast, reasoning-supervised models are substantially less stable when thinking is enabled, with their parse rates dropping significantly The instability in reasoning-supervised models is often caused by sequence-level failures: it may leak reasoning-style content into the final answer, introduce unrelated intermediate text, repeat early report sections, or stop before producing the full 14-metric review

Conclusion

The paper concludes that non-reasoning supervision consistently achieves stronger and more stable performance across both Qwen3-8B and Qwen3-4B in structured review report generation Reasoning traces are not automatically beneficial for fixed-format rubric-based evaluation, especially when the model must produce precise scores and complete structured outputs under long-context constraints Future work should investigate whether higher-quality reasoning traces, larger models, or stronger adaptation methods beyond LoRA can make reasoning supervision more effective for this setting The limitations noted include the fact that dataset construction does not involve human annotators, so the supervision signal is entirely model-generated and may contain bias and that the study only covers models in the Qwen3 family

--- Page 1 ---

When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation Jinlong Liu Mohammed Bahja Mark Lee School of Computer Science, University of Birmingham, United Kingdom jxl2069@student.bham.ac.uk;m.bahja,m.g.lee@bham.ac.uk Abstract Automatic evaluation of long-form literary writing remains challenging, as generic LLMas-Judge approaches may not fully capture creativity-related dimensions such as originality and flexibility Although the Torrance Test of Creative Writing (TTCW) provides a structured creativity framework, and prior work has demonstrated reference-based TTCW evaluation at the pairwise level, no large-scale dataset exists for long-form TTCW-based literary review generation We address this gap by constructing a dataset of 263,911 long-form stories, each annotated with scalar scores and metasynthesised review comments across 14 TTCWbased dimensions Using this dataset, we finetune Qwen3 models at two scales, 4B and 8B, under two conditions: with and without reasoning content Results show that non-reasoning fine-tuning achieves stronger and more stable performance, with the best setting reaching an evaluation score of 0.6820 Further analysis shows that reasoning-supervised models are more prone to parse failures, often continuing with irrelevant or repetitive reasoning-style text rather than completing the required 14-metric review report These results suggest that, for fixed-format rubric-based review generation, reasoning supervision is not straightforwardly beneficial, and precise metric-aligned scoring remains challenging even after task-specific fine-tuning 1 Introduction In recent years, LLM-as-Judge has become increasingly common and has shown promising reliability across multiple evaluation settings (Bonomo et al., 2025; Chiang and Lee, 2023; Liu et al., 2023) At the same time, more benchmarks and resources have been proposed for long-form literary or narrative evaluation, including ABSEVAL (Liang et al., 1Code available at https://github.com/Vince-Liuss/TTCW-based-Review 2024), STORYWARS (Du and Chilton, 2023), and COLLABSTORY (Venkatraman et al., 2025) In parallel, Chakrabarty et al. (2024) introduce TTCW as a structured creativity-oriented evaluation framework, and Li et al.

Improvements for AI systems

A dataset construction pipeline can be designed that performs metric-wise reviewer scoring, reviewer-quality filtering, and comment synthesis to produce complete TTCW review reports for long-form stories. This allows AI systems to generate structured, rubric-aligned literary reviews by leveraging multiple model evaluations and synthesizing the results.
The system can be fine-tuned specifically for non-reasoning content when generating fixed-format reviews, as the paper found that non-reasoning fine-tuning achieves stronger and more stable performance. This leads to a system that is more reliable in producing the required 14-metric report under constrained settings.
The final evaluation score can be defined as Seval = p · (0.5 sMAE + 0.5 sBERTScore-F1), which rewards models only when they are both structurally parseable and semantically close to the reference outputs. This enables a more robust measure of quality in long-form review generation tasks.
The system can be engineered to mitigate parse failures by explicitly avoiding reasoning-style content leakage, as these models were found to be more prone to parse failures, often continuing with irrelevant or repetitive reasoning-style text rather than completing the required 14-metric review report.
The model can utilize a specific set of prompts for each TTCW dimension, such as the one provided in Table 4, ensuring that all reviewer models operate under the same rubric, which improves inter-metric consistency during data generation.

When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation

Listen

Transcript

Key concepts

Terminology

Summary

Improvements for AI systems

Sources

Related papers