Lexical Hints of Accuracy in LLM Reasoning Chains

arXiv:2508.15842 · cs.CL, cs.LG · Submitted 2025-08-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Lexical Hints of Accuracy in LLM Reasoning Chains".

Jane: Here's the gist of the paper. The gist The lexical markers of uncertainty in Chain-of-Thought traces are stronger indicators of an incorrect response than high-confidence markers, supporting a lightweight post-hoc calibration signal that complements unreliable self-reported probabilities.

Tom: Fine-tuning Large Language Models (LLMs) with reinforcement learning to produce an explicit ChainofThought (CoT) before answering produces models that consistently raise overall performance on code, math, and general-knowledge benchmarks.

Jane: However, on benchmarks where LLMs currently achieve low accuracy, such as Humanity’s Last Exam (HLE), they often report high self-confidence, reflecting poor calibration.

Tom: This miscalibration masks silent failure modes and undermines the reliability of LLMs in open-ended settings.

Paper summary: Tom: So we're looking at this paper called "Lexical Hints of Accuracy in LLM Reasoning Chains". The authors are Vanhoyweghen, Verbeken, and Algaba. Basically, they're testing if the Chain-of-Thought or CoT isn't just fancy writing but actually tells us how confident the AI really is when it gives an answer.

Jane: That sounds really useful for understanding why some AI systems seem super sure about things that turn out to be wrong on harder tests, like the Humanity’s Last Exam. They claim that measuring specific parts of that thought process can give us a reliable signal about what's going on inside the model.

Lu: Exactly. They systematically analyze three feature classes: first, the length of the CoT itself; second, how much the mood changes in the thought process—what they call intra-CoT sentiment volatility; and third, these specific words used in their reasoning chains, like hedging words.

Meng: So they’re not just looking at one thing. They're breaking down the CoT into different pieces to see which piece actually works as a predictor for correctness on these challenging benchmarks. That’s a methodical approach.

Lalam: I see what they did with the datasets too; they used DeepSeek-R1 and Claude three point seven Sonnet on two types of tests: OmniMATH, where models are already pretty good, and HLE, which is much harder for them to solve correctly <ref:2508.15842#pg1,DeepSeek-R1 and Claude 3.7 Sonnet on>.

Tom: Right. And the results show that CoT length isn't a great predictor everywhere. It’s only informative on the OmniMATH benchmark because accuracy is already high there, but it carries no signal on HLE where models are struggling significantly.

Jane: That makes sense because if you're already getting seventy percent right, adding more steps doesn't help much in telling you if you’re actually right or wrong. So what do they find that *does* matter?

Lu: They found the intra-CoT sentiment volatility shows a pattern on OmniMATH, where accuracy peaks when the sentiment is consistent or slightly uplifting. But for HLE, they don't see any such pattern at all; accuracy stays uniformly low no matter how much the thought process changes in tone.

Meng: So sentiment isn't a universal signal either. It seems to work better on easier problems than on these very tough ones where the AI is already failing a lot. That tells you something about how models handle difficulty differently.

Paper summary: Lalam: Then there’s the lexical part, which they focused on hedging words—words like "guess," "stuck," or "hard." They found a negative relationship between how often these words appear and accuracy across both benchmarks.

Tom: A negative relationship means when those uncertainty markers show up more, the chance of getting the correct answer drops. They noted this trend was stronger on OmniMATH, with correlation numbers like r equals minus zero point two four for DeepSeek, and minus zero point one four for Claude three point seven Sonnet <ref:2508.15842#pg1>.

Jane: So those specific words act like little warning flags for us as humans reading their work. And they even found that models use these hedging words more when they are tackling the harder HLE benchmark, with the rate rising to about twenty percent there compared to about ten percent on OmniMATH.

Lu: And what’s interesting is how they generalized this idea. They trained a neural network on both those general features and twenty-five specific, consistently "harmful" non-lemmatized words. That combination yielded the highest Matthews correlation coefficients across both benchmarks.

Meng: From a deployment standpoint, that sounds practical if we could automate it. If we could flag an answer as potentially wrong just by checking for those top five harmful words in the CoT, that would be a lightweight filter to deploy without needing massive extra compute for full re-evaluation.

Tom: They even gave us a rule of thumb. The paper suggests you mark an answer wrong if any of those top five harmful words appear in its CoT, and otherwise classify it as correct. They reported a Matthews correlation coefficient of zero point two one five on HLE and zero point three zero five on OmniMATH with that rule.

Jane: So what does this mean for the average person who just uses these models to get quick answers? It means we don't have to trust the high confidence score they display at face value, especially on tricky questions.

Lu: The implication is that those lexical uncertainty hints actually dominate because tokens like "guess" or "stuck" can reduce the accuracy odds by up to forty percent compared to a baseline model. So, we should listen for those specific linguistic cues more than just looking at the final number.

Paper summary: Lalam: And they did point out some limitations, though. They admit that this study is focused only on English benchmarks, and they used a fixed list of hedging words that might miss uncertainty phrases in other languages.

Meng: Yeah, and they also relied on human grading for the HLE results. That means the whole signal depends on how good those human graders were at picking the right answer initially.

Tom: The paper suggests future work should involve multilingual replication with adaptive hedging lexicons and even adversarial prompting to force models to suppress those uncertainty signals entirely. It’s about pushing the limits of what these lexical signals can actually tell us.

Jane: So, for someone just listening, the big picture is that we need to look past the polished Chain-of-Thought text and start checking for specific linguistic patterns—like those words indicating doubt—to get a more honest sense of whether the AI is actually confident in its reasoning.

Lu: Exactly. The idea isn't that CoT is perfectly transparent, but that these measurable properties provide a proxy for internal confidence, which is something we can finally start measuring systematically instead of just assuming it’s fine.

Meng: For me, if this translates to production, it means we can build cheaper verification layers on top of the LLM output to catch those low-confidence cases before they cause problems down the line.

Tom: That's the practical side of it. So, we're moving from just looking at one big confidence score to analyzing a whole set of measurable signals within the AI’s generated text.

Jane: So wrapping up on "Lexical Hints of Accuracy in LLM Reasoning Chains". The authors, Vanhoyweghen, Verbeken, and Algaba, investigated whether measurable properties inside a Chain-of-Thought provide a reliable signal for an AI's internal confidence. They found that while CoT length isn't always helpful—it only predicts accuracy on OmniMATH—the intra-CoT sentiment volatility and especially specific lexical indicators like hedging words are actually strong predictors of correctness across different benchmarks. The main takeaway is that certain words, like "guess" or "stuck," can reduce the accuracy odds by up to forty percent, and this suggests we should look for these linguistic markers instead of just trusting the self-reported confidence scores.

Conclusion: Tom: So we’re wrapping up on "Lexical Hints of Accuracy in LLM Reasoning Chains." Basically, this paper is looking at whether the actual writing style of an AI's thought process gives us a better clue about its confidence than just the final confidence score it spits out.

Jane: Right. So what they did was check three different things: how long the thought process is, how much the tone shifts inside that thought process, and those specific words people use when they’re hedging—like saying "guess" or "stuck."

Lu: The key finding is that those lexical hints dominate. They found that words indicating uncertainty can reduce an answer's accuracy by up to forty percent compared to a baseline model. It’s not just about the final number; it’s about what the AI writes along the way.

Meng: From where I stand, if we could use this, it means we could build a cheaper verification layer on top of an LLM output that flags low-confidence cases before they cause problems at scale.

Tom: Exactly. The authors suggest a simple rule: if any of those top five uncertainty words appear in the thought process, you mark the answer as wrong and move on. That’s a practical filter, not some abstract theory about confidence metrics.

Jane: It shifts our focus from just trusting that high self-reported confidence score to actually analyzing the linguistic signals in the reasoning chain itself. It’s about reading between the lines of what an AI is actually doing when it’s thinking.

Lu: The limitation they mentioned is that this study was focused only on English benchmarks, and they used a fixed list of hedging words, which would miss uncertainty phrases in other languages or different contexts.

Meng: Yeah, so the next step for this research has to be multilingual replication with adaptive word lists to make it useful outside of just English math problems.

Tom: That’s where we go from theory into something deployable. We've seen how these words act as a measurable signal, but now we need to see if that signal holds up when the AI is trying to solve problems in a different language or style.

Jane: And that leads us right into how this idea could change how we evaluate and trust models across different kinds of reasoning tasks.

Data Analytics Lab, Vrije Universiteit Brussel · School of Engineering and Applied Sciences, Harvard University

cs.CL, cs.LG

Submitted: 2025-08-19

Updated: 2025-08-19

Comments: 21 pages, 7 figures, 6 tables

Code: https://github.com/Arne-Vanhoyweghen/CoT-R1-Sonnet-3.7-HLE-Omni-MATH

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 90/100

The gist: The gist The lexical markers of uncertainty in Chain-of-Thought traces are stronger indicators of an incorrect response than high-confidence markers, supporting a lightweight post-hoc calibration

Key concepts

Chain-of-Thought (CoT) Traces
These are the step-by-step reasoning sequences an LLM generates before giving a final answer. The study examines features within these traces, such as how long they are or the feeling expressed in them, to see if they can predict if the final answer is right or wrong.
Lexical Uncertainty Markers
These are specific words or phrases used in an LLM's reasoning that suggest low confidence. The researchers focused on 'hedging words,' which are linguistic cues like 'maybe' or 'stuck.' These markers were found to be the most reliable signals for predicting incorrect answers across different tasks.
OmniMATH and HLE Benchmarks
These are two large sets of tests used to evaluate the models. OmniMATH consists of many math problems where accuracy is already high, while HLE contains harder, open-ended questions that push the models' reasoning limits. The study tested how CoT features behaved on both types of challenges.

Terminology

Summary

The gist The lexical markers of uncertainty in Chain-of-Thought traces are stronger indicators of an incorrect response than high-confidence markers, supporting a lightweight post-hoc calibration signal that complements unreliable self-reported probabilities.

Introduction and Motivation

Fine-tuning Large Language Models (LLMs) with reinforcement learning to produce an explicit ChainofThought (CoT) before answering produces models that consistently raise overall performance on code, math, and general-knowledge benchmarks. However, on benchmarks where LLMs currently achieve low accuracy, such as Humanity’s Last Exam (HLE), they often report high self-confidence, reflecting poor calibration. This miscalibration masks silent failure modes and undermines the reliability of LLMs in open-ended settings. The paper investigates whether Chain-of-Thought (CoT) provides a reliable proxy for an LLM’s internal confidence by systematically analyzing three classes of features: (i) CoT length, (ii) intra-CoT sentiment volatility, and (iii) lexicographic indicators, such as hedging words.

Methodology and Datasets

The researchers collected the CoT and final responses from DeepSeek-R1 [6] and Claude Sonnet 3.7 [33] on two large-scale benchmarks of varying difficulty: OmniMATH [34] and HLE [11]. The evaluation utilized a standardized prompt requiring an Explanation, Exact Answer, and a self-reported confidence score between 0% and 100%. Omni-MATH comprises 4,428 mathematics problems ranging from basic arithmetic to advanced algebra and geometry, while HLE poses 2,088 multidisciplinary, open-ended questions that push models to the edge of their reasoning abilities. The grading for Omni-MATH was done using Omni-Judge [34].

Analysis of CoT Length and Sentiment

The study analyzed three feature classes to assess predictive power. CoT length is informative only on Omni-MATH, where accuracy is already high (≈ 70%), but carries no signal on the harder HLE (≈ 9%), indicating that CoT length predicts correctness only in the intermediate-difficulty benchmarks. Intra-CoT sentiment volatility shows a parabolic pattern for Omni-MATH, with accuracy peaking at a volatility value of approximately 0.1, suggesting that CoTs with a consistent or slightly uplifting sentiment are associated with the highest accuracy. For HLE, no such pattern is observed, where accuracy remains uniformly low regardless of the degree of sentimental variation.

Lexical Hints and Prediction Accuracy

Lexical analysis focused on hedging words, which are hypothesized to serve as cues indicating lower confidence. A negative relationship was observed between hedging rate and accuracy across benchmarks, with the trend being most pronounced on Omni-MATH where correlations were r = −0.24 (p < 0.001) for DeepSeek and r = −0.14 (p < 0.001) for Claude 3.7 Sonnet. Furthermore, hedging is more prevalent when models address the more challenging HLE benchmark, with a median rate rising to about 20% compared to about 10% for Omni-MATH. The most generalizable predictor of answer correctness was found by training a neural network on both general features and the 25 most consistently “harmful” non-lemmatized words, which yielded the highest Matthews correlation coefficients (MCC) across benchmarks.

Conclusion and Heuristic

The results reveal that lexical uncertainty hints dominate because tokens such as guess, stuck, and hard reduce accuracy odds by up to 40% relative to baseline. A lightweight rule, i.e., mark an answer wrong if any of the top 5 harmful words appears in its CoT, otherwise classify as correct, achieves MCC = 0.215 on HLE and 0.305 on Omni-Math. This suggests that a handful of carefully selected lexical cues can serve as a highly practical post-hoc filter. The contrast between models also illuminates the role of RLHF stylistics, where Claude tends to generate longer, optimism-biased CoTs yet shares the same lexical uncertainty fingerprint as R1.

Limitations and Future Work

Limitations include the study being focused on English benchmarks, a fixed hedging list that misses non-English uncertainty phrases, and the reliance on LLM–human agreement for HLE grading. Key avenues for future work include multilingual replication with adaptive hedging lexicons, adversarial prompting to force models to suppress hedging, and tracking changes in lexical signals between SFT and RLHF checkpoints. The conclusion is that a small set of words offers the most generalizable signal for predicting final response correctness across benchmarks.

--- Page 1 ---

LEXICAL HINTS OF ACCURACY IN LLM REASONING CHAINS

Arne Vanhoyweghen1,

0000-0003-0103-4715

Brecht Verbeken1

Andres Algaba1

Improvements for AI systems

  1. bold heuristic filtering of CoT responses for HLE and Omni-MATH accuracy prediction: A lightweight rule, i.e., mark an answer wrong if any of the top 5 harmful words appears in its CoT, otherwise classify as correct, achieves MCC = 0.215 on HLE and 0.305 on Omni-Math, suggesting that a handful of carefully selected lexical cues can serve as a highly practical post-hoc filter.

  2. incorporate intra-CoT sentiment volatility for better calibration signals: Analyzing the parabolic pattern observed for Omni-MATH where accuracy peaks at a volatility value of approximately 0.1 suggests that models should be calibrated based on the signed difference between the most positive and most negative sentiment within each CoT.

  3. contextual use of CoT length based on benchmark difficulty: The paper indicates that CoT length predicts correctness only for benchmarks of intermediate difficulty—that is, tasks within the model’s demonstrated capabilities but not yet saturated, implying that models should avoid relying on CoT length when assessing answers on highly challenging benchmarks like HLE.

  4. employ a hybrid prediction model utilizing multiple features: A neural network trained on general features (CoT length, emotion volatility, and hedging rate) and with the 25 most consistently “harmful” non-lemmatized words is proposed as the most generalizable predictor of answer correctness across benchmarks.

  5. track changes in RLHF artifacts for uncertainty expression: Future work should focus on tracking changes in hedging frequency and harmful token usage between SFT and RLHF checkpoints to determine if alignment techniques amplify or suppress the lexical signals that predict incorrect reasoning.

Abstract

Fine-tuning Large Language Models (LLMs) with reinforcement learning to produce an explicit Chain-of-Thought (CoT) before answering produces models that consistently raise overall performance on code, math, and general-knowledge benchmarks. However, on benchmarks where LLMs currently achieve low accuracy, such as Humanity's Last Exam (HLE), they often report high self-confidence, reflecting poor calibration. Here, we test whether measurable properties of the CoT provide reliable signals of an LLM's internal confidence in its answers. We analyze three feature classes: (i) CoT length, (ii) intra-CoT sentiment volatility, and (iii) lexicographic hints, including hedging words. Using DeepSeek-R1 and Claude 3.7 Sonnet on both Humanity's Last Exam (HLE), a frontier benchmark with very low accuracy, and Omni-MATH, a saturated benchmark of moderate difficulty, we find that lexical markers of uncertainty (e.g., guess, stuck, hard) in the CoT are the strongest indicators of an incorrect response, while shifts in the CoT sentiment provide a weaker but complementary signal. CoT length is informative only on Omni-MATH, where accuracy is already high (about 70%), and carries no signal on the harder HLE (about 9%), indicating that CoT length predicts correctness only in the intermediate-difficulty benchmarks, i.e., inside the model's demonstrated capability, but still below saturation. Finally, we find that uncertainty indicators in the CoT are consistently more salient than high-confidence markers, making errors easier to predict than correct responses. Our findings support a lightweight post-hoc calibration signal that complements unreliable self-reported probabilities and supports safer deployment of LLMs.

Sources

Related papers