Express Language Modeling
Express introduces a meta-procedure to efficiently turn high-quality unmasked attention approximations into masked versions with streaming guarantees. Thinformer Express is a new causal attention approximation offering strong theoretical bounds, constant memory usage, and sub-quadratic query time. This method significantly speeds up long-context prefill and decoding tasks while maintaining high accuracy.