Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages

arXiv:2508.10683 · cs.CL · Submitted 2025-08-14 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Neural Machine Translation for Coptic-French".

Title and authors: Tom: So we're looking at this paper today, "Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages." It’s about finding ways to actually make machine translation work when you don't have much data for an ancient language like Coptic.

Jane: Right, it tackles the problem head-on by testing several strategies. The authors are looking at things like pivot versus direct translation and how pre-training affects the final result.

Lu: They are systematically asking four main questions: should we translate directly, or use English as a middle ground? How do we pick a good model to fine-tune, and should we give the model many different versions of the same text? And finally, how do we make it strong enough to handle noise in the original texts?

Meng: That sounds like they are trying to build a really solid pipeline. From where I sit, that’s what matters when you try to ship something into production—you need resilience before you worry about fancy metrics.

Lalam: The abstract says their main finding is that fine-tuning with a training corpus that has both stylistic variation and noise awareness significantly boosts the translation quality of Coptic into French.

Tom: That’s the core idea, right? So they found that just giving the model more varied examples, even noisy ones, helps it translate better than just using a standard pre-trained multilingual model.

Jane: It sounds like they are testing two big ideas: using multiple translations for one text and deliberately injecting noise into the data to simulate how old manuscripts get damaged.

Lu: That noise simulation is interesting because it mimics real-world degradation, like missing letters or substitutions in the original Coptic texts. They found that training with fifty percent corrupted verses gave them the best trade-off between quality and robustness.

Meng: Fifty percent corruption sounds like a lot of data to clean up, but if it makes the model more resilient to real manuscript issues, I’m interested in that kind of practical benefit.

Tom: Now, when they looked at which base models to use for fine-tuning, they found that the Helsinki model performed best across all four metrics they used.

Jane: That means even though Coptic is low-resource, starting with a model like Helsinki gives you a solid foundation compared to other options like T5 or Hiero.

Lu: They noted that Hiero was also performing quite well, suggesting that knowledge learned from related languages transfers over to Coptic, which makes sense given the linguistic family connection.

Meng: So, if we’re choosing a starting point for fine-tuning, Helsinki seems like the practical winner based on their performance tests.

Tom: And when they evaluated how good the translation actually is, they used several metrics like BERTScore and COMET instead of just counting word matches.

Jane: That’s important because those semantic similarity metrics show whether the translated meaning is actually correct, not just if the words match up. They showed that models trained on specific datasets perform best when tested on that same dataset in two of those four metrics.

Lu: Interestingly, they found that training a model on three different versions of the training data consistently gave it top scores for BLEURT and COMET.

Meng: So, the strategy isn't just about one perfect dataset; it’s about combining diversity with controlled corruption to get the best overall performance across different ways of measuring quality.

Tom: The conclusion of this paper on "Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages" is that specific fine-tuning definitely beats using just pre-trained multilingual models or translating through a pivot language.

Jane: They wrapped up by saying that fine-tuning a model already trained on the ancient language or one related to it yields better results, and that training with fifty percent corrupted verses achieved the best balance for robustness.

Lu: It points toward a clear path forward: using diverse translations and noise-aware training is key for building reliable NMT systems for any low-resource or ancient language setting.

Meng: From an engineering standpoint, this gives us a concrete recipe: use Helsinki or Hiero as a base, mix in stylistic variations, inject fifty percent noise, and you get strong results.

Lalam: I agree with the practical implication. These findings extend beyond just Coptic into general guidance for building robust NMT systems for other low-resource languages out there.

Tom: So that's what we’ve got here: a systematic look at how to handle scarcity and damage in ancient language translation using specific fine-tuning techniques. That’s the whole picture on this paper.

Jane: It really shows that for these kinds of historical tasks, the way you train your model is just as important as the data itself. We’ll keep an eye on those strategies as we look at more ancient texts.

The paper's summary: Tom: So, this paper is essentially a systematic look at how to do machine translation when you have almost no data for an ancient language like Coptic into French.

Jane: It’s not just about one trick; they're testing several strategies, like whether you should translate directly or use English as a bridge language, and how much pre-training helps.

Lu: They're asking four main questions to guide the development of these tools: should we go direct or pivot? Which language model is best to fine-tune? Should we give the model lots of different versions of a passage? And how do we make it strong enough to handle noise in the original texts.

Meng: That sounds like they’re building a whole pipeline for this. I need to know if this setup actually works when you try to deploy it on something that isn't perfect.

Tom: The core finding is that fine-tuning with a training corpus that has both stylistic variation and noise awareness really boosts the translation quality.

Jane: So, they found that giving the model more varied examples, even noisy ones, helps it learn better concepts about word variations and how words are used syntactically in Coptic.

Lu: They also simulated real-world manuscript degradation by injecting noise into the text—things like missing letters or characters being substituted with similar-looking symbols.

Meng: Fifty percent corrupted verses gave them the best balance between getting good translation and keeping the model robust against those real manuscript issues. That’s a solid number for a trade-off.

Tom: They compared different starting models to see which one was best to fine-tune, and they found that the Helsinki model performed better across all their performance metrics.

Jane: Even though Coptic is low-resource, starting with a pre-trained model like Helsinki gives you a strong foundation when fine-tuning it for this specific task.

Lu: They also saw that models trained on related languages transfer well to Coptic, which makes sense given the linguistic family connection they are working with.

Tom: When they measured how good the translation actually is, they used metrics that measure semantic similarity instead of just matching words, like BERTScore and METEOR.

Jane: It’s important because these metrics show if the *meaning* is right, not just if the words line up exactly as expected. They found that models trained on a specific dataset perform best when tested on that same dataset in two of those four metrics.

Lu: Interestingly, they also saw that training a model on three different versions of the training data consistently gave it top scores for BLEURT and COMET.

Meng: So the strategy isn't just about one perfect dataset; it’s about combining diversity with controlled corruption to get the best overall performance across different ways of measuring quality.

Tom: The conclusion is that specific fine-tuning definitely beats using just pre-trained multilingual models or translating through a pivot language for these ancient languages.

Jane: They wrapped up by saying that fine-tuning a model already trained on the ancient language or one related to it yields better results, and that training with fifty percent corrupted verses achieved the best balance for robustness.

Lu: It points toward a clear path forward: using diverse translations and noise-aware training is key for building reliable machine translation systems for any low-resource or ancient language setting.

Meng: From an engineering standpoint, this gives us a concrete recipe: use Helsinki or Hiero as a base, mix in stylistic variations, inject fifty percent noise, and you get strong results.

Tom: I agree with that practical takeaway. These findings extend beyond just Coptic into general guidance for building robust machine translation systems for other low-resource languages out there.

Jane: It really shows that for these kinds of historical tasks, the way you train your model is just as important as the data itself. That leads us right into how they actually set up those noise injections and test those models in the next part.

The paper's improvements: Tom: So, after showing us the strategies they tested, they're suggesting some clear improvements for how we build these translation tools down the road.

Jane: It sounds like they’re pointing out that relying on just one perfect dataset isn't enough; we need to embrace that diversity and noise awareness more deeply in our training process.

Lu: They are emphasizing that fine-tuning with multiple different translations of the same original text is a powerful idea because it helps build a larger training corpus from scarce ancient language resources.

Meng: Having multiple versions of the same text sounds like it adds complexity to the data pipeline, but if it genuinely helps the model learn syntactic variation, I see why that's worth pursuing.

Tom: And they are pushing for a stronger focus on how noise affects performance; they want us to be more aware of how much corruption we can tolerate before the quality drops off sharply.

Jane: They suggest that while fifty percent corruption gave them the best general-case compromise, we need to understand *why* that specific level works for different types of ancient manuscripts.

Lu: They are recommending a two-pronged approach: using a model already trained on the Coptic language or one related to it, and then fine-tuning that specific model.

Meng: So it’s not just about picking the right base model; it’s about tailoring that base model specifically to the source language structure first.

Tom: And they want us to look closely at how we handle the pivot language translation—whether we should translate directly or use English as a middle ground—to see which path gives us better semantic results.

Jane: They're also suggesting that instead of just focusing on one metric, like BERTScore, we should evaluate performance across several different metrics to get a more complete picture of the quality.

Lu: They want us to realize that the best results often come from combining these techniques: diverse training data, noise simulation at a certain level, and starting with the right base model.

Meng: So if I were deploying this tomorrow, I’d start by testing those three key components—the diversity, the noise level, and the base model choice—to see what combination gives me an acceptable operational cost versus quality.

Tom: It sounds like they are moving us away from just using big multilingual models and toward more customized, specialized fine-tuning methods for low-resource languages.

Jane: That means for people who only listen to the show, it changes the expectation: we need to invest time in creating tailored training sets rather than just feeding a generic model everything we have.

Lu: The implication is that success in these areas isn't about having the biggest model; it’s about having the most informed, robust training signal possible for that specific language task.

Meng: It means we need better data curation tools if we want to scale this approach, because manually creating those diverse and noisy versions of texts is definitely not scalable for massive corpora.

Tom: It sounds like they are setting us up for the next step in research: figuring out how to automate that noise simulation and corpus generation process itself.

Conclusion: Tom: So, to wrap up the "Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages" paper, they’ve shown that specific fine-tuning is definitely better than just using general models or pivot languages.

Jane: It boils down to this: when you’re dealing with scarce resources like ancient texts, the method you use to train the AI is actually more important than just having a bigger model on hand.

Lu: They conclude that starting with a model already trained on Coptic or related languages and then fine-tuning it is the way to go for better results.

Meng: That makes sense from a deployment view; customizing the model to the specific language structure first cuts down on errors when we try to run it at scale.

Lalam: I'm quite sure of this direction; their findings show that targeted fine-tuning provides a measurable lift in translation quality for these low-resource scenarios.

Tom: They also highlighted that training with fifty percent corrupted verses gave them the best trade-off between translation quality and robustness against real manuscript damage.

Jane: That noise level, fifty percent corruption, seems to be the sweet spot for getting a resilient model without making it completely unusable.

Lu: The implication is that we should probably prioritize building systems that are inherently robust to degradation from the start rather than trying to clean everything up later with post-processing.

Meng: If we can automate that noise injection process, then this whole idea becomes much more feasible for production pipelines.

Tom: This paper provides practical guidance for anyone trying to build translation tools for historical languages or any other low-resource setting.

Jane: It’s a solid piece of work showing that systematic testing of different strategies leads to concrete, usable improvements in machine learning.

Lu: The work on "Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages" sets a good benchmark for how we approach model adaptation in the ancient language domain.

Meng: It’s interesting that they focused so heavily on the mechanics of the training signal and not just chasing some big, vague benchmark number.

Tom: We’ve seen how these kinds of structured experiments help us move from theory to something you can actually deploy reliably.

Jane: Next time we look at a paper, we'll be interested in seeing how they try to automate that noise simulation aspect they discussed today.

Nasma Chaoui, Richard Khoury

Department of Computer Science and Software Engineering, University Laval

cs.CL

Submitted: 2025-08-14

Updated: 2025-08-14

Journal ref: Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026, 482-490

DOI: 10.63317/5asa4khfm6hq

Code: https://github.com/chaouin/coptic-french-nmt

License: http://creativecommons.org/licenses/by/4.0/

The gist: Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages presents a systematic study of strategies for translating Coptic into French…

Key concepts

Pivot versus Direct Translation
This concept explores whether the best way to translate Coptic is directly into French or by using an intermediate language like English. The study compares these methods to determine which strategy yields more accurate results when developing NMT tools for ancient languages.
Multiversion Fine-Tuning
This involves training a single model on several different versions of the same original text. The hypothesis is that this helps create a larger, richer training corpus from scarce ancient language resources, allowing the model to better understand variations in syntax and word meanings.
Noise Injection
This technique simulates real-world damage to manuscripts by intentionally introducing errors like missing letters or character substitutions. Training with controlled noise helps the NMT model become robust against the degradation often found in historical texts.

Terminology

Summary

Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages presents a systematic study of strategies for translating Coptic into French, demonstrating that fine-tuning with a stylistically varied and noise-aware training corpus significantly enhances translation quality Our findings provide crucial practical insights for developing translation tools for historical languages in general

Investigating Translation Strategies

The paper systematically evaluates several key questions to guide the development of Coptic-French NMT tools, focusing on pivot versus direct translation, the impact of pre-training, the benefits of multiversion fine-tuning, and model robustness to noise The four important questions investigated are: (1) should we translate directly to our target language or use English as a pivot language; (2) how should we pick a language model to fine-tune for this task; (3) should we provide the model with a diverse set of possible translations for a passage; and (4) how can we make the model robust to noise in the original manuscript The methodology is divided into four stages, each building on the findings of the previous stage.

Building a Robust Training Signal

The study explores how to enhance translation quality through diverse training data and noise simulation The hypothesis is that having multiple different translations of a same original text will make it possible to build a larger training corpus from the scarce ancient language resource, while also allowing the NMT model to better learn concepts of syntactic variation and word synonymy Furthermore, controlled noise injection simulates real-world degradation by introducing lacunae (missing or damaged fragments) and letter substitutions (similar-looking characters). The best trade-off between translation quality and robustness was achieved when training with 50% corrupted verses

Model Selection for Fine-Tuning

The research compares different pre-trained models to determine the best choice for fine-tuning on Coptic→French translation The comparison of fine-tuned models shows that Helsinki outperforms the other models on all four metrics, attributed to its pretraining corpus including both source and target languages and being aligned with the task Hiero is also a strong performer, suggesting that knowledge learned on related languages transfers well to Coptic The T5 model achieved the worst fine-tuning results, as it did not have any Coptic nor related language in its pretraining data

Evaluating Performance Metrics

Translation quality is evaluated using metrics that measure semantic similarity rather than lexical overlap The metrics considered include BERTScore, BLEURT, COMET, and METEOR For instance, the results in Table 4 and Table 5 show that the models trained on a specific dataset translation perform best when tested on that same translation in two of our four metrics, BERTScore and METEOR Conversely, the model trained on all three dataset translations always achieves the top BLEURT and COMET scores

Conclusion on Best Practices

The study concludes that specific fine-tuning clearly improves translation quality compared to using pre-trained multilingual models or translating through a pivot language Fine-tuning a model already trained on the ancient language or a related one in the same family yields better results The best trade-off between translation quality and robustness was achieved when training with 50% corrupted verses This led to the release of two strong NMT models for this task, one based on the multilingual model of (Tiedemann et al., 2023) and the other on the Hieroglyphic translator of (Cao et al. p. 6

Where the paper says it

This led to the release of two strong NMT models for this task, one based on the multilingual model of (Tiedemann et al. , 2023) and the other on the Hieroglyphic translator of (Cao et al.

, 2024), both fine-tuned on three French versions of the training corpus with 50% noise Together, our findings extend beyond Coptic→French translation, providing practical guidance for building robust NMT systems in other low-resource or ancient language settings

Robustness to Degradation

The results in Table 7 show several clear trends regarding robustness to noise Both models trained with any level of noise perform better on clean test data, and their scores drop steadily as noise increases Models trained with less noise do better on clean data, but they are more fragile: their scores drop sharply as noise rises The optimal choice thus depends on the characteristics of the manuscript being translated, a good general-case compromise value seems to be at 50% training noise The paper presents the first systematic study of strategies for translating Coptic into French

Baseline Performance Comparison

The baseline performance comparison shows that the T5 model achieved the highest BERTScore in Table 1, with a score of 0.809 The Helsinki model achieved the highest COMET score in Table 1, with a score of 0. p. 6

Where the paper says it

What’s really happening is that models trained with less noise do better on clean data, but they’re also more fragile: their scores drop sharply as noise rises. Table 8 shows, for each model and metric, the relative score drop between 0% and 100% test noise.

248 The Hiero model achieved the highest METEOR score in Table 1, with a score of 0. p. 6
Where the paper says it

Table 8 shows, for each model and metric, the relative score drop between 0% and 100% test noise. Models trained with 0% and 10% noise lose around 20% on average—Hiero’s BLEURT score even drops by 53%.

496 These baseline results establish the performance of the four selected pre-trained models The results in Table 1 show that T5 achieved a BERTScore of 0. p. 5
Where the paper says it

521 Table 5: Comparison of dataset choice on Hiero. The results, given in Tables 4 and 5, show that the models trained on a specific dataset translation perform best when tested on that same translation in two of our four metrics, BERTScore and METEOR.

809, Helsinki achieved a BERTScore of 0. p. 4
Where the paper says it

Model BERTScore BLEURT COMET METEOR Enis 0. 119 Helsinki 0.

595, and Hiero achieved a BERTScore of 0. p. 5
Where the paper says it

521 Table 5: Comparison of dataset choice on Hiero. The results, given in Tables 4 and 5, show that the models trained on a specific dataset translation perform best when tested on that same translation in two of our four metrics, BERTScore and METEOR.

833 The results in Table 1 show that T5 achieved a BLEURT score of 0. p. 6
Where the paper says it

Table 8 shows, for each model and metric, the relative score drop between 0% and 100% test noise. Models trained with 0% and 10% noise lose around 20% on average—Hiero’s BLEURT score even drops by 53%.

501, Helsinki achieved a BLEURT score of 0. p. 4
Where the paper says it

501 Table 3: Comparison of fine-tuned models. The results translating our test corpus, presented in Table 3, show that Helsinki outperforms the other models on all four metrics.

136, and Hiero achieved a BLEURT score of 0. p. 6
Where the paper says it

Table 8 shows, for each model and metric, the relative score drop between 0% and 100% test noise. Models trained with 0% and 10% noise lose around 20% on average—Hiero’s BLEURT score even drops by 53%.

555 The results in Table 1 show that T5 achieved a COMET score of 0. p. 4
Where the paper says it

501 Table 3: Comparison of fine-tuned models. The results translating our test corpus, presented in Table 3, show that Helsinki outperforms the other models on all four metrics.

673, Helsinki achieved a COMET score of 0. p. 4
Where the paper says it

Model BERTScore BLEURT COMET METEOR Enis 0. 119 Helsinki 0.

248, and Hiero achieved a COMET score of 0. p. 6
Where the paper says it

Table 8 shows, for each model and metric, the relative score drop between 0% and 100% test noise. Models trained with 0% and 10% noise lose around 20% on average—Hiero’s BLEURT score even drops by 53%.

664 The results in Table 1 show that T5 achieved a METEOR score of 0. p. 5
Where the paper says it

521 Table 5: Comparison of dataset choice on Hiero. The results, given in Tables 4 and 5, show that the models trained on a specific dataset translation perform best when tested on that same translation in two of our four metrics, BERTScore and METEOR.

429, Helsinki achieved a METEOR score of 0. p. 4
Where the paper says it

119 Helsinki 0. 429 Hiero 0.

082, and Hiero achieved a METEOR score of 0. p. 6
Where the paper says it

Table 8 shows, for each model and metric, the relative score drop between 0% and 100% test noise. Models trained with 0% and 10% noise lose around 20% on average—Hiero’s BLEURT score even drops by 53%.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Pivot versus Direct Translation Strategy: The system can systematically compare translations produced by fine-tuned Coptic→French models against using a more general multilingual model, prompting a Coptic→English model to answer in French, or chaining two separate NMT models (Coptic→English and English→French) to determine the most accurate method.

  2. Model Selection for Fine-Tuning: The system can select the optimal pre-trained model for fine-tuning by comparing options such as taking a Coptic→English model and fine-tuning it to French, or taking a multilingual model trained for French and fine-tuning it to translate from Coptic, based on performance metrics.

  3. Dataset Enhancement through Stylistic Diversity: The system can improve translation quality by utilizing multiple different translations of the same original text during fine-tuning, hypothesizing that having multiple different translations of a same original text will make it possible to build a larger training corpus from the scarce ancient language resource.

  4. Robustness to Text Corruption: The system can be trained with simulated noise—including lacunae (missing characters), letter substitutions, and OCR errors—to build robustness against real-world manuscript degradation. Specifically, training with 50% corrupted verses was found to achieve a good general-case compromise value for resilience.

  5. Optimized Model Deployment: The system can be deployed using a model trained with 50% noise and fine-tuned on three French versions of the training corpus to achieve superior performance across both semantic similarity metrics (BERTScore and METEOR) and human quality metrics (BLEURT and COMET).

Abstract

This paper presents the first systematic study of strategies for translating Coptic into French. Our comprehensive pipeline systematically evaluates: pivot versus direct translation, the impact of pre-training, the benefits of multi-version fine-tuning, and model robustness to noise. Utilizing aligned biblical corpora, we demonstrate that fine-tuning with a stylistically-varied and noise-aware training corpus significantly enhances translation quality. Our findings provide crucial practical insights for developing translation tools for historical languages in general.

Related papers