Theoretical Analysis of Byte-Pair Encoding

arXiv:2411.08671 · cs.DS, cs.CL · Submitted 2024-11-13 · Read on arXiv

cs.DS, cs.CL

Submitted: 2024-11-13

Updated: 2026-10-30

Code: https://github.com/google/sentencepiece

Terminology

Sources

Related papers