A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models
The research introduces methods to control large language model behavior using sparse activation steering instead of dense interventions. Stochastic Token Steering (STS) and Stochastic Block Steering (SBS) use random Bernoulli gates to selectively apply a steering signal derived from a Sparse Autoencoder. Findings show that sparse steering recovers most of the full-steering effect, proving that the behavioral outcome depends on cumulative signal dosage across the sequence rather than just the number of steered tokens.