InstantHand: Feed-Forward 3D Gaussian Splatting for Animatable Hand Avatar Reconstruction from a Single Image
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "InstantHand: Feed-Forward 3D Gaussian Splatting for Animatable Hand Avatar Reconstruction from a Single Image".
Jane: Here's the gist of the paper. InstantHand is a feed-forward framework designed to reconstruct an animatable, canonical 3D Gaussian hand avatar from a single input image without requiring per-subject optimization or test-time fitting.
Tom: It leverages off-the-shelf estimates for MANO surface geometry, camera parameters, and hand masks.
Jane: The output representation is 3D Gaussian Splatting (3DGS).
Tom: The framework achieves reconstruction quality comparable to per-subject-optimized baselines while offering significant advantages in speed and zero-shot generalization.
Paper summary: Jane: So, to wrap up on "InstantHand: Feed-Forward three dee Gaussian Splatting for Animatable Hand Avatar Reconstruction from a Single Image," the authors have presented a way to reconstruct an animatable three dee Gaussian hand avatar using only one input image and relying on off-the-shelf estimates for the rest <ref:2611.08330#pg1,InstantHand: Feed-Forward 3D Gaussian Splatting for Animatable Hand Avatar Reconstruction from>.
Lu: The main thing is that they introduced two key things to solve the problem of alpha leakage in their losses, namely a Soft Alpha Band regularizer, which focuses supervision on silhouette boundaries, and Reliability-gated Appearance Transfer for refining colors using both local evidence and a global source identity prior.
Tom: And the results show that this framework achieves reconstruction quality comparable to per-subject optimized baselines while offering major advantages in speed and zero-shot generalization. They are matching the InterHand2 point 6M baseline on PSNR with twenty-six point two one dB without any test-time fitting, which is a big win for scalability.
Meng: The implication for us is that we can potentially build systems that generate high-fidelity three dee hand assets much faster and with lower inference costs by moving away from the heavy optimization steps required by traditional methods <ref:2611.08330#pg1>.
Lalam: Essentially, they’ve shown that you can separate the structure learning from the appearance conditioning, using a conditional transformer to get geometry and appearance tokens, which then combine in a way that allows for animation via MANO deformation while rendering with three dee Gaussian Splatting <ref:2611.08330#pg1>.
Jane: The paper is important because it demonstrates that high-fidelity hand avatar modeling from a single image is possible with this feed-forward approach, which makes creating animatable digital humans much more accessible than it was before.
Conclusion: Tom: So, we're done walking through the mechanics of how InstantHand reconstructs hands from one picture using three dee Gaussian Splatting. We've seen the two main training stages and those specific regularization terms like Soft Alpha Band and Reliability-gated Appearance Transfer.
Jane: It really boils down to this paper, "InstantHand: Feed-Forward three dee Gaussian Splatting for Animatable Hand Avatar Reconstruction from a Single Image." The title itself is pretty telling, isn't it? It says feed-forward and single image input.
Lu: Yeah, the core idea is getting an animatable hand avatar from just one image without needing to optimize anything per subject or test time fitting. They leverage existing estimates for surface geometry and camera parameters.
Meng: From a production standpoint, that's huge because it cuts out all that slow, iterative work we usually have to do on every new user image. If you can get the avatar in fast, you can scale way better than waiting hours for a per-subject fit.
Tom: And the results are what really stand out. They matched a per-subject optimized baseline on InterHand2 point 6M with a PSNR of twenty-six point two one versus twenty-six point one one, and they did that without any test time fitting at all <ref:2611.08330#pg2>. That's clean math for reconstruction quality.
Jane: So, the real implication here is speed and generalization. They also got competitive zero shot performance on HanCo, which means you can apply this to new hands with almost no extra training or setup cost, and the avatar acquisition cost drops by over four hundred times.
Lu: What I find interesting architecturally is the separation of roles. Stage one builds a stable prior for every Gaussian, and stage two learns exactly where the source image evidence should override that prior using a conditional transformer to generate geometry and appearance tokens.
Meng: That sounds like it’s designed to be robust against messy upstream inputs, like if the estimated hand mask or the MANO parameters are slightly off. That's something I worry about when we deploy things in a real environment where data quality isn't perfect.
Tom: It seems they’ve really nailed that separation of structure learning and appearance conditioning. They predict view independent colors rather than trying to model those tricky view dependent representations like spherical harmonics, which is a big deal for consistency across different angles.
Jane: So, to sum up, InstantHand takes a single image and spits out a high-quality three dee Gaussian representation of a hand that you can animate easily. It’s about making complex three dee assets accessible by skipping the heavy optimization steps we usually have to do in the pipeline.
Lu: And they kept it efficient too. Construction time is around two point eight four seconds on an A100, and rendering is one hundred sixty-one frames per second. It’s a solid computational footprint for what it achieves in terms of quality gain over fitting methods.
Meng: I'm still focused on the deployment costs though. The initial three point three second cost from raw image to avatar construction is manageable, but we need to make sure those upstream inputs—the MANO and camera estimates—are reliable enough for a production line, otherwise that whole system breaks down quickly at scale.
Tom: It seems like the next step is testing how well this zero shot generalization holds up when the input image quality drops significantly, which I think will tell us if this is truly robust enough for real-world use.
Chanho Kim, Muhammad Salman Ali, Kyeonghwan Gwak, Muhammad Shaheryar, Incheol Park Jeongwan On Seungryul Baek
UNIST
cs.CV
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: NeurIPS 2026. Project page: https://coolho1129.github.io/InstantHand-page/
Code: https://github.com/coolho1129/InstantHand
Project page: https://coolho1129.github.io/InstantHand-page
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 92/100
The gist: InstantHand is a feed-forward framework designed to reconstruct an animatable, canonical 3D Gaussian hand avatar from a single input image without requiring per-subject optimization or test-time…
Key concepts
- 3D Gaussian Splatting (3DGS)
- This is the output format where the hand avatar is represented as a collection of 3D Gaussians. Each Gaussian acts like a small, colored, translucent sphere that contributes to the final image when rendered. This representation allows for high-quality, view-dependent 3D reconstruction.
- Soft Alpha Band (SAB) Regularizer
- This is a specific training tool used during the first stage of learning. It focuses the model's attention specifically on the edges or silhouettes of the hand. This helps prevent thin finger details from getting overly thick during training, ensuring accurate boundary reconstruction.
- Reliability-gated Appearance Transfer
- This mechanism refines colors by mixing local evidence (what is seen in the source image) with a global prior learned from other sources. It intelligently decides how much to trust the new source color versus the established canonical color for each Gaussian, leading to more accurate final appearance.
- Feed-Forward Framework
- This describes the overall architecture where input (a single image) flows directly through a series of transformations to produce an output (the 3D avatar) in one pass. It avoids iterative optimization or complex test-time fitting, making the reconstruction process very fast and efficient.
Terminology
Summary
InstantHand is a feed-forward framework designed to reconstruct an animatable, canonical 3D Gaussian hand avatar from a single input image without requiring per-subject optimization or test-time fitting. p. 0 It leverages off-the-shelf estimates for MANO surface geometry, camera parameters, and hand masks. p. 1 The output representation is 3D Gaussian Splatting (3DGS). p. 1
The framework achieves reconstruction quality comparable to per-subject-optimized baselines while offering significant advantages in speed and zero-shot generalization. p. 1 Key architectural components include:
-
Soft Alpha Band (SAB) Regularizer: This regularizer is specifically designed to concentrate supervision on silhouette boundaries, effectively mitigating alpha leakage that can cause thin finger contours to thicken during training.
-
Reliability-gated Appearance Transfer: This mechanism refines the colors of the Gaussians by combining local evidence—transferring source colors according to each Gaussian’s rendering contribution—with a global source identity prior, which completes unobserved regions using this learned prior.
The method involves fusing source image features with MANO-surface query anchors to generate per-Gaussian tokens that parameterize a canonical Gaussian representation. p. 2 Geometry and appearance tokens are computed via a conditional transformer. p. 2 The final colors are determined by composing local evidence, the Stage-1 appearance prior, and the global source identity using reliability weighting. p. 4
The training proceeds in two stages:
-
Stage 1: Learning a stable canonical Gaussian hand representation using mask-aware photometric and perceptual losses, augmented by regularization terms like SAB.
-
Stage 2: Focusing on source-conditioned appearance modeling using source-view reconstruction, novel-view reconstruction, feature-statistic consistency (Lstat), and novel-view supervision losses.
The training data incorporates real datasets such as InterHand2. p. 66M and HO3Dv3, supplemented with synthetic source–target pairs for the appearance stage. p. 6 The model is robust to errors in upstream estimations, such as estimated masks or MANO parameters. p. 6
InstantHand demonstrates superior performance metrics: it matches the strongest per-subject-optimized baseline in PSNR (26. p. 721 dB vs 26.11 dB) on InterHand2. p. 06M without test-time fitting, and achieves competitive zero-shot performance on HanCo with a significantly lower avatar acquisition cost (over 400× lower). p. 7 Qualitative analysis confirms that InstantHand retains coherent hand geometry and fine details like wrinkles and skin texture, unlike fitted baselines which suffer from over-smoothed artifacts. p. 7
The computational efficiency is high: avatar construction takes approximately 2. p. 184 seconds on a single NVIDIA A100 (80 GB), and rendering achieves 161 FPS. p. 1 The construction time is not considered per-frame latency because it assumes upstream inputs (MANO, camera, mask) are already available; the complete raw-image-to-avatar cost is approximately 3. p. 730 seconds on the same GPU. p. 0
The framework maintains a clear separation of roles: Stage 1 establishes a stable prior for every Gaussian, while Stage 2 learns precisely where and how strongly source evidence should replace this prior, incorporating a source-level shift that propagates global appearance cues to every Gaussian. p. 4 The model predicts view-independent Gaussian colors rather than view-dependent representations like spherical harmonics. p. 5 Reliability gating is governed by its internal structure (square-root compression, median-relative thresholding), requiring no per-dataset tuning.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved reconstruction of thin, articulated structures by using a Soft Alpha Band regularizer that
concentrates supervision on silhouette boundaries.
This preventsalpha leakage which thickens thin finger contours,
leading to sharper silhouettes in novel poses. -
Enhanced appearance fidelity under single-image constraints through Reliability-gated Appearance Transfer, which
selectively enhances Gaussians supported by input observations with reliable local appearance cues to preserve input-specific details such as skin tone and fine textures.
-
Increased efficiency and speed of avatar construction by implementing a feed-forward framework that achieves
avatar construction in 2.84 seconds,
which isover 400x faster than optimization-based baselines.
-
Better handling of source appearance ambiguity by using a global source identity representation to complete unobserved regions, enabling
consistent completion under occlusion.
-
Robustness against input estimation errors by evaluating the model with estimated cues, showing that the model maintains high quality even when ground-truth source masks and MANO parameters are replaced with outputs from practical upstream estimators like SAM 3 and WiLoR.
-
Improved performance in novel-pose and novel-view rendering by using a shared representation across views, as
all views and frames are rendered from a single fixed set of canonical Gaussians,
providingcross-view and temporal consistency.
Sources
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs