NBAvatar: Neural Billboards Avatars with Realistic Hand-Face Interaction
cs.CV
Submitted: 2026-03-12
Updated: 2026-09-25
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present NBAvatar - a method for realistic rendering of head avatars handling non-rigid deformations caused by hand-face interaction.
Terminology
Abstract
We present NBAvatar - a method for realistic rendering of head avatars handling non-rigid deformations caused by hand-face interaction. To this end, we introduce a novel hybrid implicit-explicit representation for animated avatars by combining the training of explicit oriented planar primitives with implicit neural rendering. Such a combination of representations in the end-to-end pipeline enables NBAvatar to handle temporally and pose-consistent geometry, along with fine-grained appearance details provided by the neural rendering technique. To enable joint optimization of different representations we propose a geometry-aware training scheme that allows our hybrid representation to surpass existing approaches in terms of novel-view and novel-pose rendering quality. Specifically, NBAvatar achieves up to 53% LPIPS reduction compared to Gaussian-based avatar methods, while also improving PSNR and SSIM, and achieves higher structural similarity compared to the state-of-the-art hand-face interaction method InteractAvatar.
Sources
- SAM 3: Segment Anything with Concepts
- Adam: A Method for Stochastic Optimization
- Embodied Hands: Modeling and Capturing Hands and Bodies Together
- HDGS: Textured 2D Gaussian Splatting for Enhanced Scene Rendering
- PhysHand: A Hand Simulation Model with Physiological Geometry, Physical Deformation, and Accurate Contact Handling
- Frequency-Decomposed Avatar Representation for Varying Camera Distances
- Gaussian Billboards: Expressive 2D Gaussian Splatting with Textures
- DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image
- SVGS: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models