How Hard Is Tokenisation? Algorithms and Complexity Beyond NP-Hardness
cs.DS
Submitted: 2026-11-09
Updated: 2026-11-09
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- GreedTok Is a 0.3287-Approximation for Tokenization, Within 3% of the Best Possible Constant
- Tokenisation via Convex Relaxations
- OptFSST: Optimized FSST String Compression
Related papers
- Exploiting Low Scanwidth to Resolve Soft Polytomies
- PHastR: Faster Minimal Perfect Hashing with Rotated Layouts
- Fine-Grained Complexity of Approximating Vector Knapsack: A Faster Algorithm and Equivalence with k-SUM in 2D
- Polynomial-time computation of p-contraction fixed points for even p
- Separations Between Submodular-Like Graph Widths
- Bounded Integer Programming in Almost N 2d Time