GRRM: Group Relative Reward Modeling for Machine Translation
cs.CL
Submitted: 2026-02-15
Updated: 2026-09-01
Comments: Accepted by EMNLP2026; 23 pages, 7 figures
Code: https://github.com/NJUNLP/GRRM
License: http://creativecommons.org/licenses/by/4.0/
The gist: While Group Relative Policy Optimization (GRPO) offers a powerful framework for LLM post-training, its effectiveness in open-ended domains like Machine Translation hinges on accurate intra-group
Terminology
Abstract
While Group Relative Policy Optimization (GRPO) offers a powerful framework for LLM post-training, its effectiveness in open-ended domains like Machine Translation hinges on accurate intra-group ranking. We identify that standard Pointwise Quality Metrics (PQM) fall short in this context: candidates are evaluated in isolation, so the comparative context is missing for distinguishing fine-grained linguistic nuances. To address this, we introduce the Group Quality Metric (GQM) paradigm and its instantiation, the Group Relative Reward Model (GRRM). Unlike traditional independent scorers, GRRM jointly processes the entire candidate group, leveraging comparative analysis to rigorously resolve relative quality and adaptive granularity. Empirical evaluations confirm that GRRM achieves competitive ranking accuracy among all baselines; integrating GRRM into the GRPO training not only improves general translation quality but also unlocks reasoning capabilities comparable to state-of-the-art reasoning models. We release codes, datasets, and model checkpoints at https://github.com/NJUNLP/GRRM.
Sources
- Tower: An Open Multilingual Large Language Model for Translation-Related Tasks
- Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis
- Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- New Trends for Modern Machine Translation with Large Reasoning Models
- Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
- PairJudge RM: Perform Best-of-N Sampling with Knockout Tournament
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Understanding R1-Zero-Like Training: A Critical Perspective
- Generative Reward Models
- Reward Reasoning Model
- R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning
- SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translation
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Group Sequence Policy Optimization
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering