CyFA: Linear Sequence Modeling with Relative-Time-Partitioned Memory
cs.LG
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/Chyxx/CyclicFlowAttention
Sources
- Raven: High-Recall Sequence Modeling with Sparse Memory Routing
- Just read twice: closing the recall gap for recurrent language models
- Longformer: The Long-Document Transformer
- Kalman Delta Networks: Uncertainty-aware Associative Memory
- Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Log-Linear Attention
- Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
- RunningTensor: Generalizing Linear Attention to Higher-Order Recurrent States
- MDN: Parallelizing Stepwise Momentum for Delta Linear Attention
- Mistral 7B
- Kimi Linear: An Expressive, Efficient Attention Architecture
- Blurry Window Attention
- Erase-then-Delta Attention: Decoupling Erase and Write Addresses in Delta-Rule Linear Attention
- Scaling Linear Attention with Sparse State Expansion
- RWKV-7 "Goose" with Expressive Dynamic State Evolution
- DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling
- HGRN2: Gated Linear RNNs with State Expansion
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks