PERCEPT: A Corpus for POS Tagging and Analysis of Persian-English Code-Mixing

The paper introduces PERCEPT, the first publicly available large-scale Persian–English code-mixed corpus annotated with Universal Dependencies (UD) part-of-speech (POS) tags for code-mixed words and document-level topic labels. The dataset comprises 6,800 posts collected from three platforms: X (formerly Twitter), Instagram, and Digikala. The data was assembled from eight existing Persian-language datasets, with offensive content filtered out, and supplemented by 1,173 additional tweets collected via keyword searches for the 71 most frequent code-mixed words. The final dataset consists of 2,845 tweets from X, 1,463 comments from Instagram, and 1,319 comments from Digikala.

The authors propose an LLM-assisted annotation framework using Gemini 3.5 Flash to automatically identify code-mixed words, assign UPOS tags from the 17-tag Universal Dependencies tagset, and assign document-level topics from seven predefined categories: Brand & Business, Industry & Commerce, Current Events & News, Lifestyle & Personal Interests, Arts, Entertainment & Culture, Society & Identity, and Platform-Specific & Meta Topics. Platform-specific prompts were designed to exclude well-established loanwords, proper nouns, and platform-specific terms (e.g., timeline and fave star on X, story and live on Instagram, and DigiPay and DigiExpress on Digikala) from being treated as code-mixed.

Human evaluation was conducted on a random sample of 150 texts containing 268 code-mixed words, with 50 texts from each platform. Two native Persian-speaking annotators independently verified the annotations. For code-mixed token identification, the annotators achieved a precision of 81.2%, recall of 91.5%, and F1 score of 86.0%. For POS tagging, they achieved a Cohen's κ of 0.894 with 93.2% observed agreement. Comparing the LLM's annotations to the gold labels, the LLM achieved a precision of 95.5%, recall of 86.6%, and F1 score of 90.8% for token identification, and an accuracy of 97.0% with a macro F1 of 76.8% for POS tagging of correctly identified tokens. The complete annotation pipeline achieved an F1 score of 88.1%. For topic detection, the LLM achieved a Cohen's κ of 0.808 and Krippendorff's α of 0.808, with 84.7% observed agreement, a weighted F1 of 84.4%, and a macro F1 of 65.6%.

The paper presents four main empirical analyses of Persian–English code-mixing:

1. POS Distribution: NOUN is the most frequent POS category for code-mixed words across all platforms, accounting for 60.4% in X, 59.9% in Instagram, and 45.4% in Digikala. The secondary patterns vary by platform: VERB ranks second in X (23.0%), ADJ ranks second in Instagram (19.3%), and PROPN ranks second in Digikala (35.0%). The high proportion of PROPN in Digikala is attributed to frequent use of English brand names and product model names in e-commerce listings.

2. Mixing Position: The positional distribution of code-mixed words is remarkably consistent across platforms, with approximately one-third occurring in initial (35.0–37.2%), medial (35.6–35.9%), and slightly fewer in final positions (27.1–29.4%). A chi-square test found no significant association between platform and code-mixed word position (χ2(4) = 8.38, p = 0.079).

3. Triggering Effect: Measured as the proportion of utterances containing more than one code-mixed word, with length-controlled adjustment. Digikala exhibits the highest triggering effect (42.0%), followed by X (27.1%), and Instagram shows the lowest (10.2%). The higher rate in Digikala is attributed to e-commerce users frequently mentioning multiple product features, brand names, or technical specifications within a single review.

4. Topic Variation: Industry & Commerce and Brand & Business exhibit the highest mean number of code-mixed words per utterance across all three platforms. The influence of topic varies substantially across platforms: Instagram shows nearly constant code-mixing across topics (1.1–1.2 mean code-mixed words), X exhibits moderate variation, and Digikala shows the strongest topic dependence, with commerce-related discussions containing substantially more code-mixed words.

The paper concludes that code-mixing in Persian social media is predominantly realized through nouns, positional distributions are consistent across platforms, the triggering effect is most pronounced in Digikala, and commercial topics exhibit the highest degree of code-mixing. The authors note that the annotation framework is language-independent and transferable to other low-resource languages, and that PERCEPT can support the development of POS taggers and other code-mixed NLP systems, including language identification, named entity recognition, syntactic parsing, sentiment analysis, and text generation. Limitations include the specificity of findings to Persian–English code-mixing, the limited platform coverage, and the focus on POS tagging without additional annotation layers such as named entity recognition or dependency parsing.

Improvements to AI systems:

1. Code-Mixed Token Identification with Platform-Aware Filtering

• Train a language model to distinguish true code-mixed words from well-established loanwords, proper nouns, and platform-specific jargon (e.g., “timeline” on X, “story” on Instagram, “DigiPay” on Digikala).

• The improved system can automatically adapt its code-mixing detection based on the source platform, reducing false positives in social media and e-commerce text.

2. Universal Dependencies POS Tagging for Code-Mixed Text

• Fine-tune a multilingual POS tagger (e.g., using XLM-R or mT5) on the PERCEPT corpus with the 17-tag UD tagset, specifically for Persian–English mixed tokens.

• The improved system can assign accurate POS tags to code-mixed words in low-resource language pairs, enabling downstream syntactic parsing and dependency analysis without requiring separate monolingual pipelines.

3. Topic-Conditioned Code-Mixing Prediction

• Use the seven predefined topic labels (e.g., Brand & Business, Industry & Commerce) to train a model that predicts the expected degree of code-mixing per utterance.

• The improved system can pre-emptively adjust language generation or translation models—e.g., generating more English nouns for commerce-related Persian text or reducing code-mixing for lifestyle topics—based on detected topic.

4. Triggering Effect Modeling for Multi-Token Code-Mixing

• Train a sequence model to predict the probability that a code-mixed word will be followed by another code-mixed word, using the length-controlled triggering rates (42% for Digikala, 27% for X, 10% for Instagram).

• The improved system can generate more natural code-mixed text by mimicking platform-specific triggering patterns, or segment mixed utterances more accurately for language identification.

5. Position-Aware Code-Mixing Normalization

• Leverage the consistent positional distribution (initial/medial/final ≈ 35/36/29%) to train a model that predicts the likely position of code-mixed words in a sentence.

• The improved system can better handle code-mixed text in speech recognition or machine translation by reordering or aligning tokens based on expected mixing positions, improving fluency in output.

6. Cross-Lingual Transfer for Low-Resource Code-Mixed NLP

• Apply the LLM-assisted annotation framework (using Gemini 3.5 Flash) to other low-resource language pairs (e.g., Urdu–English, Hindi–English) by adapting the platform-specific prompts and exclusion lists.

• The improved system can rapidly generate annotated code-mixed corpora for new language pairs, enabling the development of POS taggers, NER, and sentiment analysis tools for under-served communities.

7. Human-in-the-Loop Annotation Refinement

• Integrate the human evaluation metrics (precision 81.2%, recall 91.5%, F1 86.0% for token ID; κ=0.894 for POS) into an active learning loop.

• The improved system can automatically flag low-confidence annotations for human review, iteratively improving its own performance on code-mixed data while reducing manual annotation effort.

Social media has become a major venue for multilingual communication, where users frequently mix multiple languages within a single utterance. Although code-mixed corpora have been developed for several language pairs, Persian-English code-mixing remains relatively underexplored. Existing Persian resources lack Universal Dependencies (UD) part-of-speech (POS) annotations for code-mixed words, limiting both linguistic analyses and the development of syntax-aware NLP models. To address this gap, we introduce PERCEPT, the first publicly available large-scale Persian-English code-mixed corpus annotated with Universal Dependencies POS tags for code-mixed words. The dataset comprises 6,800 posts collected from X, Instagram, and Digikala. We further present an LLM-assisted annotation framework that automatically assigns POS tags and document-level topics. Human evaluation demonstrates high agreement between the automatically generated annotations and gold annotations, confirming the reliability of the annotations. Using PERCEPT, we conduct the first comprehensive linguistic analysis of Persian-English code-mixing across multiple social media platforms. Our analyses reveal that nouns are the predominant category for code-mixed words, while the distributions of other POS categories vary across platforms. We further find that the positional distribution of code-mixed words is remarkably consistent across platforms, whereas the triggering effect is substantially more pronounced in Digikala. PERCEPT is publicly available at https://github.com/kalhorghazal/PERCEPT.