Programmable Sequential Memory in Coupled Oscillator Networks
summary
The gist
The paper proposes a novel sequential memory architecture based on an autonomous three-timescale dynamical system that separates stored patterns from the order in which they are visited through a…
In short
The work proposes a novel memory architecture using three coupled layers operating on different timescales to store continuous patterns and control their retrieval order. It uses a fast Kuramoto layer for storage, an intermediate hysteresis layer for dwell times, and a slow attention layer with a learnable routing matrix to sequence visits. This allows one network to realize many different traversal itineraries from the same stored patterns.
Key concepts
- Three-Timescale Dynamical System
- The system uses three distinct layers operating at different speeds: fast, intermediate, and slow. This separation is key because it allows the system to handle storage (fast), enforce reliable pauses between visits (intermediate), and manage the order of transitions (slow). This structure enables complex sequential memory operations.
- Learnable Routing Matrix K
- This matrix dictates the order in which a pattern is visited. It is learned by the system, not pre-programmed. Because it is decoupled from the physical storage weights, changing K instantly redirects the traversal path to a different sequence of patterns while keeping the stored patterns themselves unchanged.
- Continuous Pattern Encoding
- Instead of storing memories as simple binary bits, this architecture stores continuous patterns. Each pattern is encoded as an exponentially stable phase-locked configuration in the fast layer. This allows the network to store and retrieve a wide range of real-valued continuous data with high information density per stored item.
Terminology used across episodes
This episode discusses
- Programmable Sequential Memory in Coupled Oscillator Networks · Paper Radio
- A Dynamical Theory of Sequential Retrieval in Input-Driven Hopfield Networks
- Oscillator-Based Associative Memory with Exponential Capacity: Theory, Algorithms, and Hardware Implementation
The paper
Programmable Sequential Memory in Coupled Oscillator Networks · Read on arXiv
Taosha Guo, Fabio Pasqualetti
University of California, Irvine
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Programmable Sequential Memory in Coupled Oscillator Networks".
Dev: Here's the gist of the paper. The paper proposes a novel sequential memory architecture based on an autonomous three-timescale dynamical system that separates stored patterns from the order in which they are visited through a learnable routing matrix Storing and retrieving static memories in a dynamical system is by now well understood: a pattern is encoded as a stable phase-locked configuration, and recall amounts to relaxing to the nearest fixed point from a cued initial condition The main contribution of this work is the construction of a smooth, autonomous, three-timescale dynamical system that separates what is stored from in what order it is visited System Architecture The proposed architecture consists of three coupled layers operating on different timescales to manage memory storage and retrieval 1.
Rosa: A fast Kuramoto layer stores a catalog of continuous patterns as exponentially stable phase-locked configurations, each given by an exact closed-form weight vector 2.
Dev: An intermediate hysteresis layer enforces reliable dwell times between transitions 3.
Rosa: A slow attention layer routes sequential transitions via a learnable routing matrix K Encoding Memory Sequences through Phase Synchronization The storage mechanism involves assigning separate weight vectors to each catalog pattern such that the network flows downhill to it, and retrieving patterns is achieved by switching these weights The substrate is an autonomous Kuramoto network where each pattern resides as a unique exponentially stable phase-locked configuration The storage graph G nc m is constructed as a disjoint union of m cycles of length nc, with the per-cycle blocks being independent Theorem 2.1 provides a closed-form weight assignment that makes the target pattern the unique stable configuration in its winding sector Stability and Robustness Analysis The system's stability is analyzed by examining each layer individually, showing that its steady state is exponentially stable and identifying parameters for response time The fast layer exhibits exponential contraction rate κρ(α)/τθ when held in a safety margin Bρ(α) around the target configuration The intermediate recognition vector r contracts exponentially toward the running average of its input, maintaining a dwell window The slow attention layer evolves via replicator dynamics, where trajectories leave along expanding directions encoded in K to reach the successor vertex Sequential Capacity and Continuous Representation The sequential memory capacity M∗ is derived by balancing the geometry of the feasible pattern manifold F with the sharpness of the recognition softmax The bound for sequential capacity is given by M∗ ≲ π p c/γ!N−m, which grows exponentially in N − m = m(nc − 1) The Kuramoto substrate stores continuous patterns, encoding (N −m) real-valued degrees of freedom per pattern rather than N bits This continuous representation offers strictly greater information per stored pattern compared to binary sequential memory networks Context-Dependent Routing The architecture allows the traversal order to be redirected at runtime by an external context signal u(t) selecting among a library of transition rules The routing matrix K is decoupled from the physical substrate, allowing it to be learned or swapped without altering the weights A fitness signal T(u) = X S s=1 us T (s) drives the replicator layer, where T(s) are context-specific routing matrices The system can autonomously switch between different traversal graphs, such as a weekday cycle and a weekend cycle, without any transient period after the switch The paper demonstrates that separating what is stored from the order in which it is recalled allows a single network to realize qualitatively different itineraries—cyclic, branching, or terminating—without relearning the patterns themselves Acknowledgments This material is based upon work supported by the Army Research Laboratory under grant number W911NF-24-10228 References J.
Title and authors: Rosa: So we’re looking at this paper titled "Programmable Sequential Memory in Coupled Oscillator Networks". It sounds like it's tackling the problem of how systems handle sequences, not just single memories.
Dev: Right. The authors are Taosha Guo and Fabio Pasqualetti from UC Irvine, and they’re proposing something that separates what you store from the order you visit it in.
Rosa: Exactly. They’re using a three-timescale dynamical system to do this, which is a big shift because traditional Hopfield networks just store static patterns.
Dev: It suggests that instead of having one fixed way to go from memory A to memory B, you can have a separate mechanism controlling that transition flow.
Rosa: That's the core idea, separating the storage part from the routing part. It implies you could store a catalog of continuous patterns and then choose how to move between them based on what’s happening right now.
The paper's summary: Dev: So, what’s this three-timescale system actually doing in practice? They describe a fast layer for storing continuous patterns, an intermediate layer for dwell times, and a slow attention layer that routes the transitions.
Rosa: That’s right. The fast Kuramoto layer stores these patterns as exponentially stable phase-locked configurations, which is cool because it means you can store continuous patterns instead of just binary ones.
Dev: And the slow attention layer uses a learnable routing matrix to decide which pattern comes next, meaning the order isn't hard-coded into the weights anymore.
Rosa: It means you can change that order—the itinerary—without changing how the memory itself is stored in the substrate. That’s what they call decoupling storage from visit order.
Dev: It extends prior work, like Karuvally et al., by adding that intermediate layer to manage dwell times between transitions, which gives them a more precise timing mechanism.
The paper's improvements: Rosa: The authors point out several key improvements over existing models. They focus on making the storage continuous and the routing truly external to the memory structure.
Dev: They show that this approach allows for exponential storage capacity when you look at the geometry of the feasible pattern manifold F, which is a nice scaling property.
Rosa: And they mention that this continuous representation offers strictly greater information per stored pattern compared to binary sequential memory networks, because they encode real-valued degrees of freedom instead of just bits.
Dev: They also highlight context-dependent routing, where an external signal u(t) can select different transition rules, like switching between a weekday cycle and a weekend cycle.
Rosa: That context signal drives the replicator layer which selects the specific transition graph, so it’s autonomous and adapts without needing to relearn the patterns themselves.
Conclusion: Dev: So, to wrap up, this work introduces a smooth three-timescale system that separates what is stored from in what order it is visited using a learnable routing matrix.
Rosa: It means you can store continuous patterns and control the sequence of retrieval through an external context signal, allowing for flexible itineraries without changing the underlying pattern weights.
Dev: The implication is that we can build networks that realize qualitatively different sequences—cyclic, branching, or terminating—just by changing how they are routed.
Taro: I’m interested in what happens when the world misbehaves; does this system recover gracefully when it has to switch from one routing graph to another?
Rosa: That’s a key question for me. The analysis shows that the slow attention layer evolves via replicator dynamics, which means it can adapt its trajectory based on feedback from the context signal.
Dev: And they give us a formula for dwell time estimation, Tdwell = τr log one over εto l (one minus Kk,k) + τ log(one over εto l) beta (one minus Kk,k), which gives us a predictable timing for those transitions.
Taro: That predictability is important. It suggests that the system isn't just randomly switching; it’s following a learned or contextually determined path with some degree of control over the waiting time.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies