Robo-Saber: Generating and Simulating Virtual Reality Players
cs.GR, cs.AI, cs.HC, cs.LG
Submitted: 2026-02-20
Updated: 2026-09-29
Comments: 13 pages, 15 figures. Accepted to Eurographics 2026. Project page: https://robo-saber.github.io/
Journal ref: Computer Graphics Forum 45(2), e70353 (2026)
DOI: 10.1111/cgf.70353
Project page: https://robo-saber.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present the first motion generation system for playtesting virtual reality (VR) games.
Abstract
We present the first motion generation system for playtesting virtual reality (VR) games. Our player model generates VR headset and handheld controller movements from in-game object arrangements, guided by style exemplars and aligned to maximize simulated gameplay score. We train on the large BOXRR-23 dataset and apply our framework on the popular VR game Beat Saber. The resulting model Robo-Saber produces skilled gameplay and captures diverse player behaviors, mirroring the skill levels and movement patterns specified by input style exemplars. Robo-Saber demonstrates promise in synthesizing rich gameplay data for predictive applications and enabling a physics-based whole-body VR playtesting agent.
Sources
- From Sparse Signal to Smooth Motion: Real-Time Motion Generation with Rolling Prediction Models
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Generative Human Motion Stylization in Latent Space
- Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
- Berkeley Open Extended Reality Recordings 2023 (BOXRR-23): 4.7 Million Motion Capture Recordings from 105,852 Extended Reality Device Users
- Results of the 2023 Census of Beat Saber Users: Virtual Reality Gaming Population Insights and Factors Affecting Virtual Reality E-Sports Performance
- Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusion
- BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion
- CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control
- One-way communication complexity and non-adaptive decision trees
- MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space
Related papers
- SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Control
- CADReasoner: Iterative Program Editing for CAD Reverse Engineering
- QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning
- DrawVideo: Grounded and Faithful Multi-Shot Video Generation from Storyboard Keyframe Sketches
- MotionPersona: Real-Time Locomotion Control across Personas, Bodies, and Styles
- GaussianBench: Physics-Fidelity Evaluation for Gaussian Scene Representations