Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware
cs.NE, cs.LG
Submitted: 2026-08-31
Updated: 2026-08-31
Comments: 8 pages, 4 figures, Accepted at IEEE MCSOC2026
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Leveraging State Space Models in Long Range Genomics
- A Systematic Analysis of Hybrid Linear Attention
- A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations
- Rethinking the Outlier Distribution in Large Language Models: An In-depth Study
- Gaussian Error Linear Units (GELUs)
- Searching for Activation Functions
- Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark
- ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models
- ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
- Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters
- Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
- Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity
- Training-Free Activation Sparsity in Large Language Models
- Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
- Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2
- Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models
- Scalable MatMul-free Language Modeling
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
Related papers
- Evolutionary Ensemble of Agents
- Encoding and Decoding Temporal Signals with Spiking Bandpass Wavelets
- Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systems
- Evolutionary Architecture Search for Chlorophyll- a Prediction in Lakes using Sentinel-2
- Learning Alzheimer's Disease Signatures by bridging EEG with Spiking Neural Networks and Biophysical Simulations
- Neuromorphic Satellite Intelligence: A Formal Framework for Event-Driven Architectures, Spike-Based Algorithms, and Quantitative Energy-Accuracy Trade-offs