Executing Causal Structure Learning with Linear-Attention Transformers
The research constructs a fixed-weight transformer capable of exactly executing one update step of a continuous Directed Acyclic Graph (DAG) solver in logarithmic depth. This is achieved by encoding the entire state—the weighted adjacency matrix and multiplier—into the prompt. The result proves that accurate algorithm execution can be separated from accurate causal recovery, as the transformer reproduces the solver's trajectory and errors precisely.