CLOTH-HUGS: Cloth Aware Human Gaussian Splatting

summary

The gist

Cloth-HUGS presents a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing unlike prior methods that…

In short

Cloth-HUGS is a neural rendering framework using Gaussian Splatting to reconstruct photorealistic clothed humans by separating body and cloth representations into distinct Gaussian layers. It achieves this disentanglement using SMPL articulation and physics-inspired constraints, leading to more realistic garment deformation than previous methods.

Key concepts

Gaussian Splatting
A 3D neural rendering technique that represents a scene as a collection of 3D Gaussians (spherical blobs). These Gaussians define the color, density, and opacity of objects in space. Cloth-HUGS uses this to render the human body and clothing.
Disentanglement
The process of separating two or more related components—in this case, the human body and their clothing—into independent representations. Cloth-HUGS explicitly keeps these as separate Gaussian sets, which allows for better modeling of how loose clothes move independently of the underlying body structure.
Linear Blend Skinning (LBS)
A technique used in animation to deform a mesh by blending vertex positions based on weights derived from a skeletal system. Cloth-HUGS uses LBS weights specifically to control how the cloth deforms and wrinkles realistically when the body moves.
Physics-Informed Geometric Losses
Supervision techniques used during training that guide the model to produce physically plausible results. These include aligning predicted cloth geometry with ground-truth meshes (Simulation Alignment) and enforcing local structure preservation (ARAP regularization), ensuring the cloth looks structurally correct.

Terminology used across episodes

This episode discusses

The paper

CLOTH-HUGS: Cloth Aware Human Gaussian Splatting · Read on arXiv

Sadia Mubashshira, Nazanin Amini, Kevin Desai

University of Texas at San Antonio

DOI: 10.1109/ICIP61757.2026.11630175

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "CLOTH-HUGS: Cloth Aware Human Gaussian Splatting".

Jane: Here's the gist of the paper. Cloth-HUGS presents a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing unlike prior methods that absorb clothing into a single body representation and struggle with loose garments and complex deformations.

Tom: Framework Overview Cloth-HUGS represents the performer using separate Gaussian layers for body and cloth within a shared canonical space.

Jane: The canonical volume jointly encodes body, cloth, and scene primitives and is deformed through SMPLdriven articulation with learned linear blend skinning weights.

Tom: To improve cloth realism, the framework initializes cloth Gaussians from mesh topology and applies physics-inspired constraints.

Paper summary: Tom: So to summarize this paper, Cloth-HUGS is a Gaussian Splatting based neural rendering framework designed for photorealistic clothed human reconstruction that explicitly disentangles the body from the clothing.

Jane: Right, so it moves away from older methods where they absorb the clothing into a single body representation and often struggle with things like loose garments or complex deformations.

Lu: The framework represents the performer using separate Gaussian layers specifically for both the body and for the cloth within a shared canonical space.

Meng: So that means they are keeping track of two different sets of three dee representations, one for the person and one for their clothes <ref:2604.15875#pg2>? That sounds like a lot of bookkeeping.

Tom: It is bookkeeping, but it seems necessary to achieve the better performance because the canonical volume jointly encodes body, cloth, and scene primitives.

Jane: And this entire system is deformed through SMPL driven articulation using learned linear blend skinning weights to move the parts around in world space.

Lu: They initialize those body Gaussians from SMPL in a canonical pose, deriving their centers, rotations, and anisotropic scales directly from vertex positions.

Meng: So they are using the SMPL skeleton as the primary driver for the whole movement of the avatar? That’s a clear dependency there.

Tom: And for the cloth, they initialize those cloth Gaussians analogously from SNUG generated cloth meshes in a T-pose before applying physics-inspired constraints.

Jane: So they start with a known shape for the clothes and then guide it using simulation consistency, ARAP regularization, and mask supervision during training.

Lu: They also use an explicit–implicit formulation based on triplane features to encode the body and cloth properties in that shared space.

Meng: Triplane features sound like some kind of rich feature set they are using to define what the body and cloth actually look like geometrically.

Tom: Right, those triplane features feed into three decoders: an appearance decoder that outputs spherical-harmonic coefficients, a geometry decoder for corrections, and a deformation decoder for weights and offsets.

Jane: That means the geometry decoder predicts corrections to the canonical Gaussian parameters to get the deformed ones.

Lu: And the deformation decoder predicts Linear Blend Skinning weights, which are crucial because they articulate those deformed Gaussians into world space via the SMPL skeleton.

Meng: So so, if I’m thinking about deployment on a production server, what are we looking at in terms of model size and training complexity?

Tom: The paper claims they introduce a depth-aware multi-pass rendering pipeline to handle occlusions between the body, cloth, and scene for robust body–cloth–scene compositing.

Jane: That pipeline handles the blending between the body Gaussians and the cloth Gaussians using spherical-harmonic coefficients during rendering.

Lu: The final image is obtained by combining these passes, so it’s a way to handle things like virtual try-on or cloth editing effectively.

Conclusion: Tom: So to wrap up this paper, Cloth-HUGS tackles the problem of creating photorealistic clothed human reconstruction by explicitly modeling cloth as independent geometric entities.

Jane: The authors are Sadia Mubashshira, Nazanin Amini from the University of Texas at San Antonio.

Lu: It’s important to remember that they achieved this by combining three dee Gaussian Splatting with physics-inspired refinement and depth-aware rendering techniques <ref:2604.15875#pg2,refinement and depth-aware rendering>.

Meng: So for us in production, what’s the implication? Is this something we can actually deploy quickly or is it just a high-end research tool?

Tom: The authors state that this method enables plausible cloth deformation that follows body motion while maintaining the efficiency of three dee Gaussian Splatting <ref:2604.15875#pg2>.

Jane: And they validated this by showing that well-regularized LBS weights provide strong geometric guidance for realistic clothed human rendering, especially on datasets like NeuMan and ZJU-MoCap.

Lu: Ultimately, the framework successfully achieves state-of-the-art performance across diverse motions, reducing LPIPS by twenty-eight percent on average on NeuMan three while consistently improving PSNR and SSIM <ref:2604.15875#pg2,achieves state-of-the-art performance across diverse motions, reducing LPIPS by>.

Meng: So what does this mean for the real world? Does it mean we can actually render realistic clothes that move naturally in video?

Tom: It means we have a framework that can produce cloth deformation that follows body motion plausibly, which is significant for applications like virtual try-on or interactive visualization.

More episodes

← Home