CLOTH-HUGS: Cloth Aware Human Gaussian Splatting
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "CLOTH-HUGS: Cloth Aware Human Gaussian Splatting".
Jane: Here's the gist of the paper. Cloth-HUGS presents a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing unlike prior methods that absorb clothing into a single body representation and struggle with loose garments and complex deformations.
Tom: Framework Overview Cloth-HUGS represents the performer using separate Gaussian layers for body and cloth within a shared canonical space.
Jane: The canonical volume jointly encodes body, cloth, and scene primitives and is deformed through SMPLdriven articulation with learned linear blend skinning weights.
Tom: To improve cloth realism, the framework initializes cloth Gaussians from mesh topology and applies physics-inspired constraints.
Paper summary: Tom: So to summarize this paper, Cloth-HUGS is a Gaussian Splatting based neural rendering framework designed for photorealistic clothed human reconstruction that explicitly disentangles the body from the clothing.
Jane: Right, so it moves away from older methods where they absorb the clothing into a single body representation and often struggle with things like loose garments or complex deformations.
Lu: The framework represents the performer using separate Gaussian layers specifically for both the body and for the cloth within a shared canonical space.
Meng: So that means they are keeping track of two different sets of three dee representations, one for the person and one for their clothes <ref:2604.15875#pg2>? That sounds like a lot of bookkeeping.
Tom: It is bookkeeping, but it seems necessary to achieve the better performance because the canonical volume jointly encodes body, cloth, and scene primitives.
Jane: And this entire system is deformed through SMPL driven articulation using learned linear blend skinning weights to move the parts around in world space.
Lu: They initialize those body Gaussians from SMPL in a canonical pose, deriving their centers, rotations, and anisotropic scales directly from vertex positions.
Meng: So they are using the SMPL skeleton as the primary driver for the whole movement of the avatar? That’s a clear dependency there.
Tom: And for the cloth, they initialize those cloth Gaussians analogously from SNUG generated cloth meshes in a T-pose before applying physics-inspired constraints.
Jane: So they start with a known shape for the clothes and then guide it using simulation consistency, ARAP regularization, and mask supervision during training.
Lu: They also use an explicit–implicit formulation based on triplane features to encode the body and cloth properties in that shared space.
Meng: Triplane features sound like some kind of rich feature set they are using to define what the body and cloth actually look like geometrically.
Tom: Right, those triplane features feed into three decoders: an appearance decoder that outputs spherical-harmonic coefficients, a geometry decoder for corrections, and a deformation decoder for weights and offsets.
Jane: That means the geometry decoder predicts corrections to the canonical Gaussian parameters to get the deformed ones.
Lu: And the deformation decoder predicts Linear Blend Skinning weights, which are crucial because they articulate those deformed Gaussians into world space via the SMPL skeleton.
Meng: So so, if I’m thinking about deployment on a production server, what are we looking at in terms of model size and training complexity?
Tom: The paper claims they introduce a depth-aware multi-pass rendering pipeline to handle occlusions between the body, cloth, and scene for robust body–cloth–scene compositing.
Jane: That pipeline handles the blending between the body Gaussians and the cloth Gaussians using spherical-harmonic coefficients during rendering.
Lu: The final image is obtained by combining these passes, so it’s a way to handle things like virtual try-on or cloth editing effectively.
Conclusion: Tom: So to wrap up this paper, Cloth-HUGS tackles the problem of creating photorealistic clothed human reconstruction by explicitly modeling cloth as independent geometric entities.
Jane: The authors are Sadia Mubashshira, Nazanin Amini from the University of Texas at San Antonio.
Lu: It’s important to remember that they achieved this by combining three dee Gaussian Splatting with physics-inspired refinement and depth-aware rendering techniques <ref:2604.15875#pg2,refinement and depth-aware rendering>.
Meng: So for us in production, what’s the implication? Is this something we can actually deploy quickly or is it just a high-end research tool?
Tom: The authors state that this method enables plausible cloth deformation that follows body motion while maintaining the efficiency of three dee Gaussian Splatting <ref:2604.15875#pg2>.
Jane: And they validated this by showing that well-regularized LBS weights provide strong geometric guidance for realistic clothed human rendering, especially on datasets like NeuMan and ZJU-MoCap.
Lu: Ultimately, the framework successfully achieves state-of-the-art performance across diverse motions, reducing LPIPS by twenty-eight percent on average on NeuMan three while consistently improving PSNR and SSIM <ref:2604.15875#pg2,achieves state-of-the-art performance across diverse motions, reducing LPIPS by>.
Meng: So what does this mean for the real world? Does it mean we can actually render realistic clothes that move naturally in video?
Tom: It means we have a framework that can produce cloth deformation that follows body motion plausibly, which is significant for applications like virtual try-on or interactive visualization.
Sadia Mubashshira, Nazanin Amini, Kevin Desai
University of Texas at San Antonio
cs.CV
Submitted: 2026-04-17
Updated: 2026-04-17
Journal ref: 2026 IEEE International Conference on Image Processing (ICIP), 2026
DOI: 10.1109/ICIP61757.2026.11630175
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 79/100
The gist: Cloth-HUGS presents a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing unlike prior methods that…
Key concepts
- Gaussian Splatting
- A 3D neural rendering technique that represents a scene as a collection of 3D Gaussians (spherical blobs). These Gaussians define the color, density, and opacity of objects in space. Cloth-HUGS uses this to render the human body and clothing.
- Disentanglement
- The process of separating two or more related components—in this case, the human body and their clothing—into independent representations. Cloth-HUGS explicitly keeps these as separate Gaussian sets, which allows for better modeling of how loose clothes move independently of the underlying body structure.
- Linear Blend Skinning (LBS)
- A technique used in animation to deform a mesh by blending vertex positions based on weights derived from a skeletal system. Cloth-HUGS uses LBS weights specifically to control how the cloth deforms and wrinkles realistically when the body moves.
- Physics-Informed Geometric Losses
- Supervision techniques used during training that guide the model to produce physically plausible results. These include aligning predicted cloth geometry with ground-truth meshes (Simulation Alignment) and enforcing local structure preservation (ARAP regularization), ensuring the cloth looks structurally correct.
Terminology
Summary
Cloth-HUGS presents a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing unlike prior methods that absorb clothing into a single body representation and struggle with loose garments and complex deformations. p. 0
Framework Overview
Cloth-HUGS represents the performer using separate Gaussian layers for body and cloth within a shared canonical space. The canonical volume jointly encodes body, cloth, and scene primitives and is deformed through SMPLdriven articulation with learned linear blend skinning weights. p. 0 To improve cloth realism, the framework initializes cloth Gaussians from mesh topology and applies physics-inspired constraints. p. 0 The authors introduce a depth-aware multi-pass rendering strategy for robust body–cloth–scene compositing. p. 0
Representation and Deformation
The framework disentangles the clothed human into body and cloth Gaussian sets in a shared canonical space. The body Gaussians are initialized from SMPL in the canonical pose, with centers, rotations, and anisotropic scales derived from vertex positions. p. 1 Cloth Gaussians are initialized analogously from SNUG-generated cloth meshes in a T-pose. p. 1 Body and cloth properties are encoded using an explicit–implicit formulation based on triplane features. p. 1 The geometry decoder predicts position, rotation, and scale corrections yielding deformed Gaussian parameters. p. 1
Training Signal Construction
The training process incorporates several physics-inspired supervision techniques to guide the learning of Gaussian parameters. The combined loss function includes L1, SSIM, and LPIPS losses alongside cloth-specific regularization terms. p. 4 Key supervision components include:
-
Cloth LBS Regularization to regularize predicted cloth Linear Blend Skinning weights using reference weights transferred from the SMPL body. p. 2
-
Physics-Informed Geometric Losses, which include Simulation Alignment (Lsim) to align predicted cloth geometry with SNUG-generated meshes. p. 2
-
ARAP Regularization (LARAP) to preserve local cloth structure by applying an As-Rigid-As-Possible constraint. p. 2
-
Mask Consistency (Lmask) to enforce silhouette consistency between rendered and ground-truth cloth masks. p. 2
Performance and Validation
Experiments on multiple benchmarks show that Cloth-HUGS improves perceptual quality and geometric fidelity over state-of-the-art baselines. p. 0 The method reduces LPIPS by up to 28% while producing temporally coherent cloth dynamics. p. 0 Quantitative results on NeuMan show Cloth-HUGS achieving a lower average LPIPS of 0. p. 3113 compared to 0. p. 3185. Furthermore, the method reduces FID from 68. p. 1693 to 64.757 on full scenes. p. 4 The results demonstrate that well-regularized LBS weights provide strong geometric guidance for realistic clothed human rendering. p. 4
Conclusion
The paper concludes by stating that the method enables plausible cloth deformation that follows body motion while retaining the efficiency of 3D Gaussian Splatting. The integration of cloth priors from SNUG with neural rendering validates that well-regularized LBS weights provide strong geometric guidance for realistic clothed human rendering. p. 4 The framework successfully achieves state-of-the-art performance across diverse motions, reducing LPIPS by 28% on average on NeuMan [3] while consistently improving PSNR, SSIM, and FID. p. 1 The overall results confirm that body–cloth disentanglement and physics-guided regularization are essential for photorealistic clothed human reconstruction. p. 4
References
[1] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng
[2] Chung-Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. p. 5 Barron, and Ira Kemelmacher-Shlizerman
[3] Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, and Anurag Ranjan
[4] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuhler, ¨ and George Drettakis
[5] Muhammed Kocabas, Jen-Hao Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan
[6] Shoukang Hu and Ziwei Liu
[7] Jiahui Lei, Yufu Wang, Georgios Pavlakos, Lingjie Liu, and Kostas Daniilidis
[8] Chaitanya Patel, Zhouyingcheng Liao, and Gerard PonsMoll <ref:2604.15875#pg4>
[9] Igor Santesteban, Nils Thuerey, Miguel A Otaduy, and Dan Casas <ref:2604. p. 515875#pg4>
[10] Igor Santesteban, Miguel A. Otaduy, and Dan Casas <ref:2604.15875#pg4>
[11] Yang Zheng, Qingqing Zhao, Guandao Yang, Yifan Wang, Donglai Xiang, Florian Dubost, Dmitry Lagun, Thabo Beeler <ref:2604.15875#pg4>
[12] Kangkan Wang, Chong Wang, Jian Yang <ref:2604.15875#pg4>
[13] Kangkan Wang, Guofeng Zhang, Suxu Cong, and Jian Yang <ref:2604.15875#pg4>
[14] Chen Guo, Tianjian Jiang, Manuel Kaufmann, Chengwei Zheng <ref:2604.15875#pg4>
[15] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black <ref:2604. p. 015875#pg4>
[16] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li <ref:2604.15875#pg4>
[17] Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai <ref:2604.15875#pg4>
[18] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter <ref:2604.15875#pg4>
[19] Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang <ref:2604.15875#pg4>
[20] Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron <ref:2604.15875#pg4>
[21] Chen Guo, Tianjian Jiang, Xu Chen, Jie Song, and Otmar Hilliges <ref:2604.15875#pg4>
[22] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuhler, ¨ and George Drettakis <ref:2604.15875#pg4>
[23] Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai <ref:2604.15875#pg4>
[24] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll <ref:2604.15875#pg4>
[25] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang <ref:2604.15875#pg4>
[26] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuhler, ¨ and George Drettakis <ref:2604.15875#pg4>
[27] Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai <ref:2604.15875#pg4>
[28] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll <ref:2604.15875#pg4>
[29] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang <ref:2604.15875#pg4>
--- Page 1 ---
CLOTH-HUGS: CLOTH AWARE HUMAN GAUSSIAN SPLATTING Sadia Mubashshira, Nazanin Amini, Kevin Desai University of Texas at San Antonio, San Antonio, TX, USA ABSTRACT We present Cloth-HUGS, a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing Unlike prior methods that absorb clothing into a single body representation and struggle with loose garments and complex deformations. Cloth-HUGS represents the performer using separate Gaussian layers for body and cloth within a shared canonical space The canonical volume jointly encodes body, cloth, and scene primitives and is deformed through SMPLdriven articulation with learned linear blend skinning weights. p. 0 To improve cloth realism, we initialize cloth Gaussians from mesh topology and apply physics-inspired constraints. p. 0 We further introduce a depth-aware multi-pass rendering strategy for robust body–cloth–scene compositing. p. 0 Experiments on multiple benchmarks show that Cloth-HUGS improves perceptual quality and geometric fidelity over state-of-the-art baselines.
How the training signal is built
The framework integrates physics-guided supervision, including simulation alignment, as-rigid-as-possible (ARAP) regularization, and cloth-specific linear blend skinning (LBS) constraints. The authors further introduce a depth-aware multi-pass rendering strategy for robust body–cloth–scene compositing. p. 0 The combined loss is defined as L = Lrec + λcloth-lbsLcloth-lbs + λsimLsim + λARAPLARAP + λmaskLmask, with Lrec = λL1LL1 + λSSIMLSSIM + λLPIPSLLPIPS. p. 2
Where the gains come from
The primary gains stem from explicitly disentangling body and cloth into separate canonical Gaussian layers. The depth-aware multi-pass rendering pipeline handles occlusions between body, cloth, and scene. p. 1 Cloth Gaussians are initialized and supervised using pseudo-ground-truth cloth geometry generated by SNUG [10]. p. 0 These components work together to enable photorealistic clothed human synthesis with controllable garment dynamics and efficient training. p. 0
Evaluation Metrics
The authors report PSNR, SSIM, and LPIPS to measure pixel fidelity, structural similarity, and perceptual quality. They also report FID to quantify distributional differences between rendered and real images. p. 2 The results show that Cloth-HUGS achieves higher PSNR, comparable SSIM, and substantially lower LPIPS on full scenes. p. 4
Cloth-HUGS introduces a neural rendering framework that explicitly models cloth as independent geometric entities while maintaining skeletal coupling through LBS weight regularization. The method enables plausible cloth deformation that follows body motion while retaining the efficiency of 3D Gaussian Splatting. p. 4 Experiments on NeuMan [3] and ZJU-MoCap [17] datasets validate that well-regularized LBS weights provide strong geometric guidance for realistic clothed human rendering. p. 4 The final results demonstrate improved perceptual realism over prior Gaussian-based methods. p. 4
--- Page 2 ---
Canonical Space Cb, Ob Depth-Aware Compositing SNUG Pose Θ Physics-Informed Geometric Losses LBS Splat Camera pose Scene Triplane Rendered Frame Human Gaussians in world coordinate MLPs DA Canonical Space DG DD DA DG DD Cc, Oc Pseudo ground-truth cloth mesh Cloth Decoder Body Decoder Fig. 1. Cloth-HUGS Overview. p. 1 From a monocular video, we build a body–cloth disentangled Gaussian avatar using SMPL-conditioned TriPlane features, physics-inspired cloth refinement, and depth-aware rendering. p. 1
The body Gaussians are initialized from SMPL in the canonical pose, with centers µB i, rotations RB i, and anisotropic scales sB i derived from vertex positions. The cloth Gaussians are initialized analogously from SNUG-generated cloth meshes Mc i in a T-pose. p. 1 Body and cloth are assigned separate triplane encodings in a canonical space. p. 1 The appearance decoder DA outputs spherical-harmonic coefficients cSH ∈ R 16×3 and opacity o = σ(ˆo). p. 1 p. 1
Deformation Mechanism
The geometry decoder DG predicts Gaussian position, rotation, and scale corrections yielding deformed Gaussian parameters. The deformation decoder DD predicts Linear Blend Skinning weights w ∈ R 24 (softmaxnormalized) and pose-dependent offsets ∆p ∈ R 207×3, modeling non-rigid effects such as muscle deformation and cloth wrinkles. p. 2 p. 2 The predicted weights articulate µdef to world space via the SMPL skeleton. p. 2
Depth-Aware Compositing
The depth-aware multi-pass rendering pipeline is employed to handle occlusions between body, cloth, and scene. In the first pass, body and scene Gaussians are jointly rendered using standard alpha blending to produce a base image Ibase. p. 2 In the second pass, cloth Gaussians are rendered to obtain Icloth using spherical-harmonic (SH) coefficients. p. 2 The final image is obtained as Ifinal = Icloth ⊙ Vcloth + Ibase ⊙ (1 − Vcloth). p. 2
--- Page 3 ---
TriPlane Feature Decoding and Deformation Given triplane features ftri(p), three MLP decoders predict Gaussian appearance, geometry, and deformation attributes. The appearance decoder DA outputs spherical-harmonic coefficients cSH ∈ R 16×3 and opacity o = σ(ˆo). The geometry decoder DG predicts position, rotation, and scale corrections (∆µ, ∆r, ∆s), yielding deformed Gaussian parameters µdef = µcanon + ∆µ. p. 2 The deformation decoder DD predicts Linear Blend Skinning weights w ∈ R 24 (softmaxnormalized) and pose-dependent offsets ∆p ∈ R 207×3, modeling non-rigid effects such as muscle deformation and cloth wrinkles.
Cloth-based Supervision
Cloth LBS Regularization regularizes predicted cloth Linear Blend Skinning (LBS) weights using reference weights transferred from the SMPL body. Ground-truth weights Wgt cloth are obtained by mapping cloth vertices in T-pose to nearest SMPL vertices. p. 2 The loss is Lcloth-lbs = λcloth-lbs∥Wcloth − Wgt cloth∥2. p. 2
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Cloth-aware Gaussian Splatting framework: This system explicitly disentangles body and cloth into separate Gaussian layers in a shared canonical space, enabling cloth specific motion, deformation, and depth layering while retaining SMPL driven animatability.
-
Physics guided supervision: The integration of physics inspired losses including simulation alignment, as rigid as possible (ARAP) regularization, and cloth specific linear blend skinning (LBS) constraints improves temporal stability and realism by imposing physical consistency on the cloth geometry.
-
Depth aware multi pass rendering: A depth aware multi pass rendering pipeline handles body cloth scene occlusions through a two pass process to produce a final image using the formula Ifinal = Icloth ⊙ Vcloth + Ibase ⊙ (1 − Vcloth), ensuring correct cloth occlusion while allowing semi transparent blending.
-
Controllable garment dynamics: The system enables controllable garment dynamics and efficient training, allowing applications such as cloth editing and virtual try on by explicitly modeling loose garments through separate cloth Gaussians supervised by pseudo ground truth cloth mesh from SNUG.
Sources
- HUGS: Human Gaussian Splats
- GART: Gaussian Articulated Template Models
- HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs