LoRi: Low-Rank Distillation for Implicit Reasoning
LoRi proposes a low-rank distillation framework to transfer reasoning from explicit chain-of-thought into compact implicit latent processes. It aligns teacher and student trajectories in a shared low-rank subspace using first and second-order statistics, effectively compressing long reasoning paths into shorter latent representations while maintaining high accuracy.