Mamba Hawkes Process for Event Sequence Modeling
cs.LG, stat.ML
Submitted: 2024-07-07
Updated: 2026-10-29
License: http://creativecommons.org/licenses/by/4.0/
The gist: Modeling asynchronous event sequences is crucial in numerous real-world applications such as healthcare monitoring, financial transaction analysis, and so on.
Terminology
Abstract
Modeling asynchronous event sequences is crucial in numerous real-world applications such as healthcare monitoring, financial transaction analysis, and so on. Traditional temporal point processes, including Hawkes Processes, often fail to capture complex dependencies due to their parametric limitations. While neural approaches like RNNs and Transformers have improved flexibility, they struggle with computational inefficiency, and attention saturation. In this paper, we introduce the Mamba Hawkes Process (MHP), the first framework to integrate selective state space model (Mamba) with temporal point processes. MHP leverages time-varying state transitions and input-dependent gating to efficiently encode event history and capture long-term dependencies with linear complexity. Importantly, we provide theoretical guarantees showing that MHP generalizes both classical multi-exponential Hawkes processes and exponential-decay gated RNNs, underscoring its expressive power and theoretical soundness. To address the inherent constraints of pure state space models in handling heterogeneous event interactions, we further develop Adaptive Mamba Hawkes Process (A-MHP) that incorporates two novel mechanisms: a Time-Scaling Mechanism that adaptively weights time intervals based on event type and history, and a Dual-Channel State Transition that adaptively processes event content and temporal dynamics for more refined state updates. Extensive experiments on synthetic and realworld datasets demonstrate that MHP and A-MHP consistently outperform state-of-the-art baselines in event prediction tasks, particularly in long-sequence scenarios. Our work establishes a scalable and theoretically grounded paradigm for event sequence modeling, with practical implications for predictive maintenance, anomaly detection, and dynamic system analysis.
Sources
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Hungry Hungry Hippos: Towards Language Modeling with State Space Models
- RoTHP: Rotary Position Embedding-based Transformer Hawkes Process
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Efficiently Modeling Long Sequences with Structured State Spaces
- Hawkes Processes
- Jamba: A Hybrid Transformer-Mamba Language Model
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning