VWEN: A Curated Small Language Model for Vietnamese Paraphrasing
cs.CL
Submitted: 2026-11-11
Updated: 2026-11-11
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present VWEN, a 0.8B-parameter model for Vietnamese paraphrasing at a requested level of change and for five related tasks: generating several paraphrases, simplification, gist…
Abstract
We present VWEN, a 0.8B-parameter model for Vietnamese paraphrasing at a requested level of change and for five related tasks: generating several paraphrases, simplification, gist, register transfer and paraphrase judgement. VWEN is Qwen3.5-0.8B fine-tuned on about 7k samples chosen by a four-stage pipeline: rules and deduplication, a bidirectional entailment filter, a divergence score that assigns low and high paraphrase levels, and selection for coverage and instruction-following difficulty. We compare it with paraphrase-only recipes from the same pipeline and with training on the ViSP corpus under matched and larger step budgets. On high-divergence paraphrasing, VWEN reaches a success rate (requested level reached and meaning preserved) of 0.61 against 0.46 for the ViSP model at matched compute, a difference that holds on inputs resembling ViSP. Given four times more training examples, the ViSP model closes the gap on such inputs but not on questions, which ViSP lacks and which it often turns into statements. Across all comparisons the entailment filter is the most effective stage. Training on paraphrases alone erases abilities of the base model that the multi-task data preserves.
Sources
- A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamese
- Multilingual E5 Text Embeddings: A Technical Report
- Qwen3 Technical Report
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data