Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning
The study investigates whether function-vector attention heads in large language models act as a single unit or are split into two opposing roles: 'writers' and 'cancellers.' Using sign-preserving causal analysis, researchers found that heads with positive effects favor the correct task label (writers), while negative effects oppose it (cancellers). These roles are task-conditioned and exhibit distinct mechanistic properties.