Does Native 3D Texture Generation Necessarily Require 3D Assets for Training?
cs.CV
Submitted: 2026-09-28
Updated: 2026-09-28
Comments: Project Page: https://github.com/wangjiangshan0725/Tex-Zero
Code: https://github.com/wangjiangshan0725/Tex-Zero
License: http://creativecommons.org/licenses/by/4.0/
The gist: Native 3D texture generation synthesizes colors directly in 3D space for a given geometry, conditioned on multi-view reference images.
Terminology
Abstract
Native 3D texture generation synthesizes colors directly in 3D space for a given geometry, conditioned on multi-view reference images. It is generally believed that training such models requires large-scale, high-quality real 3D asset data, whose acquisition remains a long-standing and challenging problem. In this work, we propose Tex-Zero, demonstrating that a high-fidelity native 3D texture generation framework can be trained without 3D assets. Our key observation is that only high-quality and fine-grained color information is essential for 3D texture training, while the required geometric information is less critical and can be manually constructed rather than obtained from real 3D assets. This finding makes it possible to transform abundant, high-quality 2D images into effective training samples for 3D texture generation. Specifically, we convert high-quality 2D images into 3D training samples by representing each image as a plane in 3D space and applying patch-wise random rotations and aggregation to construct complex geometric structures. Using these constructed image data, we train the Tex-Zero VAE, which can reconstruct real 3D assets with high quality despite never observing them during training. Building upon the Tex-Zero VAE, we train the Tex-Zero DiT also exclusively on the constructed image data, where the conditioning 2D multi-view images are transformed into planes in 3D space and also encoded by the Tex-Zero VAE, thereby reducing the representation gap and improving generation quality. Extensive experiments show that Tex-Zero generates high-fidelity 3D textures with fine-grained details solely using images as training data, offering a promising perspective on the data paradigm for scaling 3D texture generation.
Sources
- Text2Tex: Text-driven Texture Synthesis via Diffusion Models
- BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset
- MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
- VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models
- Hitem3D 2.0: Multi-View Guided Native 3D Texture Generation
- MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusion
- TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resampling
- Segment Anything
- NaTex: Seamless Texture Generation as Latent Color Diffusion
- UniTEX: Universal High Fidelity Generative Texturing for 3D Shapes
- Flow Matching for Generative Modeling
- Text-Guided Texturing by Synchronized Multi-View Diffusion
- TEXTure: Text-Guided Texturing of 3D Shapes
- Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material
- Qwen-Image Technical Report
- Native and Compact Structured Latents for 3D Generation
- Structured 3D Latents for Scalable and Versatile 3D Generation
- LRM-Zero: Training Large Reconstruction Models with Synthesized Data
- TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splatting
- TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models