Investigating Single-Block Recurrence in Vision Transformers for Image Recognition
Researchers investigated a single-block recurrent Vision Transformer (bViT) by applying one transformer block repeatedly instead of stacking independent blocks. They found that sufficient embedding width allows this recurrence to recover most of the performance of standard ViTs, suggesting a shared block implicitly encodes multiple step-dependent computations.