Tangut Word Segmentation under Extreme Resource Scarcity: Integrating Traditional Lexicons and Unlabeled Text
The research tackled automatic word segmentation for Tangut, a language with limited resources, by combining expert annotations with traditional dictionaries and unlabeled text using a unified BIES–CRF framework. The core finding is that integrating contextual pretraining via the TangutEncoder yields the best performance (mean F1 of 0.911), showing strong generalization beyond the limited labeled vocabulary.