System-Prompt Conditioning and Hidden-State Geometry in Four Open-Weight Models: Corrections and What Survives
The study investigated how identity prompts affect geometric representations in transformer hidden states across four models with different training regimes. It found that the way identity is encoded changes based on training; in base models, it's direction-based, but in instruction-tuned models, it shifts to magnitude. This reorganization is specific to multimodal instruction tuning and relates to how identity information survives geometric transformations.