Scaling Legal AI: Benchmarking Mamba and Transformers for Statutory Classification and Case Law Retrieval
cs.CY, cs.AI, cs.LG
Submitted: 2025-08-29
Updated: 2026-10-07
Terminology
Sources
- LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
- Longformer: The Long-Document Transformer
- Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
- Efficiently Modeling Long Sequences with Structured State Spaces
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- Mamba Retriever: Utilizing Mamba for Effective and Efficient Dense Retrieval
- Repeat After Me: Transformers are Better than State Space Models at Copying
- When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset
- SaulLM-7B: A pioneering Large Language Model for Law
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- Generating Long Sequences with Sparse Transformers
- Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents
- Jamba: A Hybrid Transformer-Mamba Language Model
- Jamba-1.5: Hybrid Transformer-Mamba Models at Scale
- Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
- Zamba: A Compact 7B SSM Hybrid Model
- Hymba: A Hybrid-head Architecture for Small Language Models
Related papers
- Reasoning Enhances Robustness to Prompt Injection in LLM-Based Consensus
- Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot
- PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks
- What is an intelligent system?
- AI University: An LLM-Powered Learning Assistant for Engineering---A Finite Element Method Case Study
- Generative AI Use in Entrepreneurship: An Integrative Review and an Empowerment-Entrapment Framework