Learning to Group and Order: Cross-Instance Self-Supervised RL for Vision-Centric MLLMs
cs.CV
Submitted: 2026-10-31
Updated: 2026-11-04
Comments: Accepted to NeurIPS 2026
Code: https://github.com/SEU-VIPGroup/GO-SSL
License: http://creativecommons.org/licenses/by/4.0/
The gist: Reinforcement learning (RL) post-training has become an important paradigm for improving multimodal large language models (MLLMs), especially with automatically verifiable rewards.
Terminology
Abstract
Reinforcement learning (RL) post-training has become an important paradigm for improving multimodal large language models (MLLMs), especially with automatically verifiable rewards. Self-supervised visual pretext tasks provide such rewards without human annotations. However, existing methods mainly exploit within-instance structure, offering limited supervision for cross-instance discrimination and fine-grained comparison. We introduce Group-and-Order Self-Supervised Reinforcement Learning (GO-SSL), a cross-instance jigsaw framework for vision-centric MLLM post-training. Given two visual instances, GO-SSL mixes their local elements and trains the model to group them by instance of origin while recovering the order within each group, coupling instance-level comparison with spatial, temporal, or geometric structure recovery. This paired formulation turns the same unlabeled data into richer verifiable supervision through diverse pairings, shuffles, and a random-to-hard curriculum. We instantiate GO-SSL mainly on cross-image jigsaw tasks and further extend it to 3D depth and video temporal ordering. Across diverse benchmarks, GO-SSL consistently improves over the base MLLM and prior self-supervised post-training baselines, with clear gains in fine-grained perception and spatial understanding. Further evaluations across model scales and families support its generality. These results suggest that cross-instance comparison provides a useful self-supervised RL signal for improving transferable visual understanding in MLLMs. Code is available at https://github.com/SEU-VIPGroup/GO-SSL.
Sources
- Qwen2.5-VL Technical Report
- OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Unsupervised Representation Learning by Predicting Image Rotations
- SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning
- BEiT: BERT Pre-Training of Image Transformers
- ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
- LLaVA-Video: Video Instruction Tuning With Synthetic Data
- DINOv2: Learning Robust Visual Features without Supervision
- Seeing the Arrow of Time in Large Multimodal Models
- TempCompass: Do Video LLMs Really Understand Videos?
- Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
- Static and Plugged: Make Embodied Evaluation Simple
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs