Yo-ByT5: Efficient and High-Fidelity Diacritic Restoration for Yor`ub'a

arXiv:2610.01634 · cs.CL · Submitted 2026-10-01 · Read on arXiv

cs.CL

Submitted: 2026-10-01

Updated: 2026-10-01

Comments: 7 pages, 3 figures, 3 tables. Code and outputs: https://github.com/lazy-monster/yo-byt5

License: http://creativecommons.org/licenses/by/4.0/

The gist: Yorùbá is a widely spoken tonal language that depends on diacritics to avoid lexical ambiguity.

Terminology

Abstract

Yorùbá is a widely spoken tonal language that depends on diacritics to avoid lexical ambiguity. However, it is often written without these diacritics, thereby hindering downstream Natural Language Processing (NLP) tasks. In this paper, we introduce Yo-ByT5, a byte-level Automatic Diacritic Restoration (ADR) model fine-tuned from ByT5-small. We evaluate Yo-ByT5 alongside five publicly released Yorùbá ADR models and one open-weight large language model (LLM) on the YAD benchmark under a consistent protocol. Our results demonstrate that Yo-ByT5 matches the performance of the strongest existing model, mT5-base, with a DER of 10.14% and a CER of 3.48%. Furthermore, it exhibits superior text fidelity despite using approximately half the parameter count of mT5-base. We also release our training code and model outputs, as well as call for the development of a larger, purpose-built benchmark for Yorùbá diacritic restoration.

Sources

Related papers