Understanding Transformer-based Vision Models through Inversion
The study introduces a novel, modular feature inversion method to efficiently reconstruct images from intermediate representations of large vision transformers like DETR and ViT. By training smaller, localized inverse components instead of one large network per layer, the method reveals how these models gradually change representations across layers and highlights systematic differences in how they preserve image details and color information.