RouTex: Multi-reference 3D Texture Generation by Reprogramming Rotary Position Encoding
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "RouTex: Multi-reference 3D Texture Generation by Reprogramming Rotary Position Encoding".
Jane: Here's the gist of the paper. While single-image 3D texture generation has advanced significantly, conditioning texture synthesis on multi-view reference images remains largely under-explored In this paper, we propose RouTex, a novel multi-reference 3D texture generation framework based on in-context Diffusion Transformers (DiT) Our key insight is that native 3D Rotary Position Encoding (RoPE) inherently possesses spatial addressing capabilities We introduce Positional Routing (PR) to reprogram 3D RoPE along spatial and semantic dimensions, establishing explicit reference-to-view correspondence without introducing additional parameters To execute this routing efficiently, Dual-Granularity Attention (DGA) combines per-view local attention for fine texture fidelity with global joint attention over deduplicated references for crossview consistency Extensive experiments demonstrate that RouTex achieves new state-of-the-art performance in multireference 3D texture generation Core Mechanism of Reference Alignment The framework leverages the inherent spatial addressing capabilities of 3D Rotary Position Encoding (RoPE) by introducing Positional Routing (PR) to reprogram it without adding parameters.
Tom: This reprogramming involves two complementary strategies: 1.
Jane: Coordinate-Space Routing, which divides the 360◦ camera yaw into four canonical azimuth regions and aligns reference spatial coordinates with those of the first target view in the same region.
Paper summary: Tom: So, to wrap up this paper on RouTex, it’s about taking three dee textures and conditioning them on multiple reference images using this new framework based on in-context Diffusion Transformers <ref:2611.06070#pg1>.
Jane: It basically says they figured out how to use existing components—specifically the three dee Rotary Position Encoding—to handle multiple views without adding a ton of extra parameters to the model <ref:2611.06070#pg2>.
Lu: The authors introduce something called Positional Routing, which reprograms that position encoding along both spatial and semantic dimensions so it can explicitly link each reference image to its target view <ref:2611.06070#pg2>.
Meng: So they’re saying you don't need a whole new set of learned weights just to manage the different camera angles, but you still get the texture consistency across them.
Tom: Exactly, and they use Dual-Granularity Attention to handle the attention efficiently—local attention for fine detail and global joint attention for cross-view consistency <ref:2611.06070#pg3>.
Jane: The main result is that RouTex achieves state-of-the-art performance in generating three dee textures when you have many reference inputs, which is a big deal because most existing methods struggle with more than one reference <ref:2611.06070#pg3>.
Lu: They show through ablation studies that removing the positional routing or the dual attention completely tanks the quality, so those parts are really necessary for this multi-reference capability <ref:2611.06070#pg3>.
Tom: And the training setup involves sampling how many references you use, say between one and four, and it requires at least one reference from a specific spatial region <ref:2611.06070#pg3>.
Jane: What this means for us is that if you’re working on high-quality three dee asset generation where consistency across different camera angles matters, this gives you a concrete way to condition the synthesis on several views at once.
Lu: It moves the focus from just generating one perfect view to generating a coherent texture that makes sense when viewed from multiple perspectives simultaneously <ref:2611.06070#pg3>.
Tom: So they’ve shown a framework where you use inherent properties of the position encoding and clever attention design to solve this multi-reference alignment problem in three dee texture generation <ref:2611.06070#pg3>.
Jane: That leaves us wondering how robust this method is when you move beyond textures and try to apply it to other types of complex three dee scene synthesis.
Conclusion: Tom: So, we’re wrapping up on RouTex now, and to summarize the title: "RouTex: Multi-reference three dee Texture Generation by Reprogramming Rotary Position Encoding."
Jane: Right, so that name tells you exactly what they did—they took three dee textures and made them consistent across multiple reference images by reprogramming a specific part of the AI model called Rotary Position Encoding.
Lu: The authors are doing this inside a framework built on in-context Diffusion Transformers, or DiT models. That’s the engine driving the synthesis.
Meng: So what does that actually mean for someone trying to get better texture generation? Does it just give them a fancier name for something they already do?
Tom: No, it’s about the mechanism. They are showing you how to use what's already there—the inherent spatial addressing in the three dee Rotary Position Encoding—to explicitly manage multiple reference inputs without adding a ton of new parameters to the model.
Jane: That’s the big deal. It suggests you don't need a whole new set of learned weights just to make sure different camera angles look like they belong together on one surface.
Lu: They achieve this through Positional Routing, which basically re-maps those positions based on where the reference image is in terms of space and meaning.
Tom: And they pair that with Dual-Granularity Attention, which handles the efficiency so you get fine detail locally while keeping the global consistency across all those different views.
Jane: The conclusion is that RouTex gives us a concrete, working method for high-quality three dee texture generation when you have several reference images to choose from. It’s a specific engineering solution to a problem in representation learning.
Lu: It proves that by cleverly reprogramming the encoding and using smart attention tricks, you can achieve state-of-the-art results for multi-reference setups.
Meng: So if we want to improve texture quality using multiple reference images, we need something that explicitly handles the alignment between those views, like this Positional Routing.
Tom: Exactly. It’s a concrete framework for doing that—leveraging the existing structure of the position encoding and then routing it with PR and DGA to solve that multi-reference alignment problem in three dee texture generation.
Chengyang Li, Yang Liu, Jiale Zhang, Qiang Rao, Wei Zhang, *Taicheng Huang
JD.com
cs.CV
Submitted: 2026-11-04
Updated: 2026-11-04
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 93/100
The gist: While single-image 3D texture generation has advanced significantly, conditioning texture synthesis on multi-view reference images remains largely under-explored In this paper, we propose RouTex…
Key concepts
- 3D Rotary Position Encoding (RoPE)
- This is a mathematical way of encoding spatial information in 3D space, similar to how models understand where things are. The paper shows that this existing encoding can be reprogrammed without adding new parameters, allowing the model to better understand the relationship between different views and references spatially.
- Positional Routing (PR)
- This technique modifies the 3D RoPE to create a direct connection between a reference image and a target view. It works by dividing camera angles into regions and using formulas based on candidate view pools to explicitly define which reference corresponds to which target view, establishing clear spatial alignment.
- Dual-Granularity Attention (DGA)
- This mechanism manages computation efficiently by using two attention stages. The first stage handles local attention for fine texture details within each view-reference pair. The second stage performs global joint attention across all views and references to ensure the final texture looks consistent across all inputs.
- Flux-Kontext Architecture
- This is the underlying model structure that RouTex is built upon. It has a native architecture that allows for efficient implementation of the Dual-Granularity Attention mechanism, which helps in balancing local detail preservation and global cross-view coherence during texture synthesis.
Terminology
Summary
While single-image 3D texture generation has advanced significantly, conditioning texture synthesis on multi-view reference images remains largely under-explored In this paper, we propose RouTex, a novel multi-reference 3D texture generation framework based on in-context Diffusion Transformers (DiT) Our key insight is that native 3D Rotary Position Encoding (RoPE) inherently possesses spatial addressing capabilities We introduce Positional Routing (PR) to reprogram 3D RoPE along spatial and semantic dimensions, establishing explicit reference-to-view correspondence without introducing additional parameters To execute this routing efficiently, Dual-Granularity Attention (DGA) combines per-view local attention for fine texture fidelity with global joint attention over deduplicated references for crossview consistency Extensive experiments demonstrate that RouTex achieves new state-of-the-art performance in multireference 3D texture generation
Core Mechanism of Reference Alignment
The framework leverages the inherent spatial addressing capabilities of 3D Rotary Position Encoding (RoPE) by introducing Positional Routing (PR) to reprogram it without adding parameters. This reprogramming involves two complementary strategies:
-
Coordinate-Space Routing, which divides the 360◦ camera yaw into four canonical azimuth regions and aligns reference spatial coordinates with those of the first target view in the same region. p. 1
-
Fractional Semantic Addressing, which reprograms the semantic axis d to distinguish token types, views, and reference regions using formulas involving candidate view pool size V and number of azimuth regions R. p. 1
Efficient Attention Execution
Dual-Granularity Attention (DGA) is employed to manage the computational efficiency of routing by exploiting the native architecture of Flux-Kontext This mechanism utilizes two stages:
Double-Stream Blocks (Per-View Local Attention):
The double-stream stage isolates each target view into an independent sequence, allowing attention to be restricted to each view-reference pair by reshaping the batch dimension.
Single-Stream Blocks (Global Joint Attention):
Before the single-stream stage, reference features are merged into K unique representations via scatter-averaging, allowing joint attention across all target views and unique references to promote cross-view appearance consistency.
Training and Evaluation Protocol
The model is trained using the vanilla flow-matching loss The training setup involves several specific parameters:
Reference Count:
The number of reference images K is randomly sampled from the set of 1, 2, 3, or 4, strictly requiring at least one reference from region 0.
View Pool:
The view pool consists of 30 candidate viewpoints, including six canonical orthographic views and twenty-four surrounding views.
Target Sampling:
Six target views are randomly sampled per iteration while ensuring full coverage of all four azimuth regions.
Performance and Necessity Validation
Experimental results confirm that RouTex outperforms all baselines, showing improvements in metrics like FID and KID compared to methods using single or limited multi-reference inputs. Ablation studies validate the necessity of each component: removing PR or DGA leads to significant performance degradation, while removing both components results in a substantial drop in quality. p. 3 The qualitative results further show that RouTex faithfully reproduces complex textures across all input views, unlike single-reference methods which generate flat surfaces. p. 2
Conclusion
In this work, we present a multi-reference framework for high-quality 3D texture generation within a 2D multi-view diffusion framework We introduce Positional Routing (PR) to establish explicit reference-to-view correspondence through reprogrammed 3D RoPE, and Dual-Granularity Attention (DGA) to efficiently combine local texture transfer with global cross-view consistency With these components, RouTex achieves state-of-the-art performance in multi-reference 3D texture generation.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Positional Routing (PR) reprograms 3D RoPE to align each reference’s spatial coordinates with its corresponding target view,
enabling self-attention to automatically route appearance guidance.
This allows for parameter-free appearance routing based on coarse azimuth categories rather than precise camera calibration. -
Dual-Granularity Attention (DGA) combines per-view local attention for fine texture fidelity and global joint attention over deduplicated references for crossview consistency, achieving
high-fidelity texture
by preserving local details at a lower compute cost. -
The improved system can generate high-quality 3D textures conditioned on multiple reference images by establishing explicit reference-to-view correspondence, which is crucial for resolving surface ambiguity in unobserved regions.
Sources
- MVDream: Multi-view Diffusion for 3D Generation
- FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space
- Flow Matching for Generative Modeling
- TexVerse: A Universe of 3D Objects with High-Resolution Textures
- Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs