Lexical Hints of Accuracy in LLM Reasoning Chains
Researchers investigated whether Chain-of-Thought (CoT) traces reliably predict an LLM's final answer correctness by analyzing CoT length, sentiment, and specific words. Findings show that lexical uncertainty markers, like hedging words such as 'guess' or 'stuck,' are stronger indicators of incorrect responses than high-confidence markers. A simple rule based on these top five harmful words can effectively filter wrong answers.