The Dichotomy Between Pattern Recognition and Step-by-Step Reasoning
The paper explores how Large Language Models (LLMs) choose between step-by-step reasoning and pattern recognition based on context length. It investigates how De Bruijn structure in training data influences this choice, finding that moderate structure balances accuracy and robustness against small changes. The study confirms that reasoning with frequent state emission is more accurate for fewer samples.