Three tiers of computation in transformers and in brain architectures
The study mapped human linguistic and logical abilities onto a hierarchy of formal grammar models (G-A) to test transformer models. It found that while large models excel at lower tiers like context-free grammar, they struggle with higher tiers like context-sensitive grammars. The research suggests that scaling size is less important than the architectural transition between these computational levels for achieving advanced reasoning capabilities.