Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention

arXiv:2605.08453 · cs.LG, cs.AI, stat.ML · Submitted 2026-05-08 · Read on arXiv

cs.LG, cs.AI, stat.ML

Submitted: 2026-05-08

Updated: 2026-11-03

Code: https://github.com/karpathy/char-rnn

Project page: https://attention-motifs.github.io/v2/index.html

Terminology

Sources

Related papers