CLOTH-HUGS: Cloth Aware Human Gaussian Splatting
summary
The gist
Cloth-HUGS presents a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing unlike prior methods that…
In short
Cloth-HUGS is a neural rendering framework using Gaussian Splatting to reconstruct photorealistic clothed humans by separating body and cloth representations into distinct Gaussian layers. It achieves this disentanglement using SMPL articulation and physics-inspired constraints, leading to more realistic garment deformation than previous methods.
Key concepts
- Gaussian Splatting
- A 3D neural rendering technique that represents a scene as a collection of 3D Gaussians (spherical blobs). These Gaussians define the color, density, and opacity of objects in space. Cloth-HUGS uses this to render the human body and clothing.
- Disentanglement
- The process of separating two or more related components—in this case, the human body and their clothing—into independent representations. Cloth-HUGS explicitly keeps these as separate Gaussian sets, which allows for better modeling of how loose clothes move independently of the underlying body structure.
- Linear Blend Skinning (LBS)
- A technique used in animation to deform a mesh by blending vertex positions based on weights derived from a skeletal system. Cloth-HUGS uses LBS weights specifically to control how the cloth deforms and wrinkles realistically when the body moves.
- Physics-Informed Geometric Losses
- Supervision techniques used during training that guide the model to produce physically plausible results. These include aligning predicted cloth geometry with ground-truth meshes (Simulation Alignment) and enforcing local structure preservation (ARAP regularization), ensuring the cloth looks structurally correct.
Terminology used across episodes
This episode discusses
- CLOTH-HUGS: Cloth Aware Human Gaussian Splatting · Paper Radio
- HUGS: Human Gaussian Splats
- GART: Gaussian Articulated Template Models
- HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields
The paper
CLOTH-HUGS: Cloth Aware Human Gaussian Splatting · Read on arXiv
Sadia Mubashshira, Nazanin Amini, Kevin Desai
University of Texas at San Antonio
DOI: 10.1109/ICIP61757.2026.11630175
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "CLOTH-HUGS: Cloth Aware Human Gaussian Splatting".
Jane: Here's the gist of the paper. Cloth-HUGS presents a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing unlike prior methods that absorb clothing into a single body representation and struggle with loose garments and complex deformations.
Tom: Framework Overview Cloth-HUGS represents the performer using separate Gaussian layers for body and cloth within a shared canonical space.
Jane: The canonical volume jointly encodes body, cloth, and scene primitives and is deformed through SMPLdriven articulation with learned linear blend skinning weights.
Tom: To improve cloth realism, the framework initializes cloth Gaussians from mesh topology and applies physics-inspired constraints.
Paper summary: Tom: So to summarize this paper, Cloth-HUGS is a Gaussian Splatting based neural rendering framework designed for photorealistic clothed human reconstruction that explicitly disentangles the body from the clothing.
Jane: Right, so it moves away from older methods where they absorb the clothing into a single body representation and often struggle with things like loose garments or complex deformations.
Lu: The framework represents the performer using separate Gaussian layers specifically for both the body and for the cloth within a shared canonical space.
Meng: So that means they are keeping track of two different sets of three dee representations, one for the person and one for their clothes <ref:2604.15875#pg2>? That sounds like a lot of bookkeeping.
Tom: It is bookkeeping, but it seems necessary to achieve the better performance because the canonical volume jointly encodes body, cloth, and scene primitives.
Jane: And this entire system is deformed through SMPL driven articulation using learned linear blend skinning weights to move the parts around in world space.
Lu: They initialize those body Gaussians from SMPL in a canonical pose, deriving their centers, rotations, and anisotropic scales directly from vertex positions.
Meng: So they are using the SMPL skeleton as the primary driver for the whole movement of the avatar? That’s a clear dependency there.
Tom: And for the cloth, they initialize those cloth Gaussians analogously from SNUG generated cloth meshes in a T-pose before applying physics-inspired constraints.
Jane: So they start with a known shape for the clothes and then guide it using simulation consistency, ARAP regularization, and mask supervision during training.
Lu: They also use an explicit–implicit formulation based on triplane features to encode the body and cloth properties in that shared space.
Meng: Triplane features sound like some kind of rich feature set they are using to define what the body and cloth actually look like geometrically.
Tom: Right, those triplane features feed into three decoders: an appearance decoder that outputs spherical-harmonic coefficients, a geometry decoder for corrections, and a deformation decoder for weights and offsets.
Jane: That means the geometry decoder predicts corrections to the canonical Gaussian parameters to get the deformed ones.
Lu: And the deformation decoder predicts Linear Blend Skinning weights, which are crucial because they articulate those deformed Gaussians into world space via the SMPL skeleton.
Meng: So so, if I’m thinking about deployment on a production server, what are we looking at in terms of model size and training complexity?
Tom: The paper claims they introduce a depth-aware multi-pass rendering pipeline to handle occlusions between the body, cloth, and scene for robust body–cloth–scene compositing.
Jane: That pipeline handles the blending between the body Gaussians and the cloth Gaussians using spherical-harmonic coefficients during rendering.
Lu: The final image is obtained by combining these passes, so it’s a way to handle things like virtual try-on or cloth editing effectively.
Conclusion: Tom: So to wrap up this paper, Cloth-HUGS tackles the problem of creating photorealistic clothed human reconstruction by explicitly modeling cloth as independent geometric entities.
Jane: The authors are Sadia Mubashshira, Nazanin Amini from the University of Texas at San Antonio.
Lu: It’s important to remember that they achieved this by combining three dee Gaussian Splatting with physics-inspired refinement and depth-aware rendering techniques <ref:2604.15875#pg2,refinement and depth-aware rendering>.
Meng: So for us in production, what’s the implication? Is this something we can actually deploy quickly or is it just a high-end research tool?
Tom: The authors state that this method enables plausible cloth deformation that follows body motion while maintaining the efficiency of three dee Gaussian Splatting <ref:2604.15875#pg2>.
Jane: And they validated this by showing that well-regularized LBS weights provide strong geometric guidance for realistic clothed human rendering, especially on datasets like NeuMan and ZJU-MoCap.
Lu: Ultimately, the framework successfully achieves state-of-the-art performance across diverse motions, reducing LPIPS by twenty-eight percent on average on NeuMan three while consistently improving PSNR and SSIM <ref:2604.15875#pg2,achieves state-of-the-art performance across diverse motions, reducing LPIPS by>.
Meng: So what does this mean for the real world? Does it mean we can actually render realistic clothes that move naturally in video?
Tom: It means we have a framework that can produce cloth deformation that follows body motion plausibly, which is significant for applications like virtual try-on or interactive visualization.
More episodes
- 2603.28113-The multiplicative Lipschitz bound as an inductive bias
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound