MACRO: Training-free Multi-plane Attention for Closeup Render Optimization

arXiv:2607.03875 · cs.CV · Submitted 2026-07-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "MACRO: Training-free Multi-plane Attention for Closeup Render Optimization".

Jane: Here's the gist of the paper. Close-up rendering from 3D Gaussian splatting remains an open challenge because recent diffusion-based methods produce significant artifacts in this setting The features in reference-conditioned enhancement models are not scale-invariant, causing cross-view attention to retrieve incorrect correspondences when the same content appears at different scales Building on this analysis we introduce MACRO, Multi-plane Attention for Closeup Render Optimization, a training-free method for high-quality close-up novel view synthesis from 3DGS The Failure of Existing Enhancement Methods Recent diffusion-based methods enhance renderings by conditioning on real reference images from the training set to transfer genuine textures and details However, in close-up settings these methods produce poor results because they assume that the target and reference views observe the scene at a similar scale Existing approaches that do target close-up synthesis attempt to bridge the gap gradually but do not tackle the actual root cause: objects in the close-up and reference are at different scales The key insights identified are that VAE and U-Net features used in diffusion-based enhancement are sensitive to scale, causing attention to retrieve incorrect correspondences Furthermore the VAE encoder is not scale-equivariant, which prevents correction in latent space and necessitates image-space solutions How the Training Signal is Built MACRO resolves the scale mismatch by leveraging known 3D structure to decompose the close-up into depth planes The process involves three main components: 1.

Tom: Multi-plane depth decomposition: The close-up rendering is decomposed into P depth planes using k-means clustering on the rendered depth map Dcu to group pixels at similar depths 2.

Jane: Scale-matched reference cropping: For each plane p we compute a scale-matched crop of each reference image that shows the corresponding scene content at the same scale as it appears in the close-up view This involves calculating a scale factor sp and cropping the reference image to cover a region of 1/sp of its field of view before resizing it 3.

Paper summary: Tom: So, what’s their actual solution? They introduce MACRO, which they call Multi-plane Attention for Closeup Render Optimization. It’s a training-free method that uses known three dee structure to fix this scale mismatch <ref:2607.03875#pg1>.

Jane: How does it actually work without retraining anything? It seems like they decompose the close-up rendering into different depth planes using something called k-means clustering on the rendered depth map.

Lu: That’s right, they do a multi-plane depth decomposition, which breaks the close-up down into several distinct layers based on how deep the pixels are.

Meng: And for each of those planes, they compute a scale-matched crop from the reference images so that what you see in that plane matches the scale of your close-up view perfectly.

Lalam: So, it’s not just one reference image anymore; different depth planes get their own perfectly scaled reference crops to ensure the attention mechanism only looks at things that match its specific scale.

Tom: The big payoff here is that they show this works on top of existing enhancement models, like Difix, and they claim it improves perceptual metrics by five percent to nearly thirty percent over the best enhancement baselines.

Jane: That sounds significant because it’s improving the quality of these close-ups without needing any new training data or architectural changes to the core system.

Lu: The analysis showed that by applying a depth-aware attention mask, each closeup token only attends to reference tokens at the matching scale, which is what allows this multi-plane approach to work effectively.

Meng: From an engineering standpoint, if you can apply this as a preprocessing step rather than retraining the whole diffusion model, that’s much cheaper and faster for deployment.

Conclusion: Tom: So we’ve covered how MACRO addresses the scale mismatch in close-up rendering by decomposing it into depth planes and using scale-matched attention to handle the reference images.

Jane: It seems the authors, Nitzan Hodos and Roy Amoyal, are arguing that since the VAE encoder isn't scale-equivariant, image space solutions for this scaling issue are necessary.

Lu: Exactly. The paper concludes that MACRO successfully restores details that were previously missing because those details couldn't be recovered from the original three dee representation alone <ref:2607.03875#pg1>.

Meng: What does this mean for deployment? It suggests we don't need to constantly worry about scale inconsistencies when dealing with novel view synthesis in close-up applications, as long as we use a method like MACRO.

Lalam: The implication is that they’ve provided a practical way to get faithful close-ups from three dee Gaussian splatting without needing extensive retraining cycles for the enhancement part <ref:2607.03875#pg1>.

Tom: So the title really captures it: Multi-plane Attention for Closeup Render Optimization, showing how this specific attention strategy solves the scale invariance problem we identified earlier.

Nitzan Hodos, Roy Amoyal, Lior Fritz, Ianir Ideses, Sagie Benaim, Netalee Efrat

Amazon Prime Video 2Technion Israel Institute of Technology Ben Gurion University Hebrew University of Jerusalem

cs.CV

Submitted: 2026-07-04

Updated: 2026-11-14

Comments: Project page: https://nitzanhod.github.io/MACRO

Project page: https://nitzanhod.github.io/MACRO

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 91/100

The gist: Close-up rendering from 3D Gaussian splatting remains an open challenge because recent diffusion-based methods produce significant artifacts in this setting The features in reference-conditioned…

Key concepts

3D Gaussian Splatting (3DGS)
This is a 3D representation technique that uses a collection of 3D points (Gaussians) with associated colors and opacities to model scenes. It's used for rendering novel views, but close-up rendering remains challenging because existing enhancement methods struggle with the scale differences between the close-up and reference images.
Diffusion-based Enhancement
These are methods that use diffusion models to improve renderings by conditioning them on real reference images. They transfer details from the reference image to the target rendering. However, these methods fail in close-up settings because they assume both views see the scene at similar scales, leading to incorrect feature matching.
Multi-plane Depth Decomposition
This process breaks down a single close-up rendering into several distinct depth planes. It uses k-means clustering on the rendered depth map to group pixels that appear at similar distances from the camera. This decomposition is crucial for MACRO to handle the varying scales present in a close-up view.
Scale-matched Reference Cropping
This step ensures that reference images are cropped and resized to match the specific scale of each depth plane identified during decomposition. By calculating a scale factor and cropping reference views appropriately, MACRO aligns the content across different scales, allowing for accurate cross-view attention.

Terminology

Summary

Close-up rendering from 3D Gaussian splatting remains an open challenge because recent diffusion-based methods produce significant artifacts in this setting The features in reference-conditioned enhancement models are not scale-invariant, causing cross-view attention to retrieve incorrect correspondences when the same content appears at different scales Building on this analysis we introduce MACRO, Multi-plane Attention for Closeup Render Optimization, a training-free method for high-quality close-up novel view synthesis from 3DGS

The Failure of Existing Enhancement Methods

Recent diffusion-based methods enhance renderings by conditioning on real reference images from the training set to transfer genuine textures and details However, in close-up settings these methods produce poor results because they assume that the target and reference views observe the scene at a similar scale Existing approaches that do target close-up synthesis attempt to bridge the gap gradually but do not tackle the actual root cause: objects in the close-up and reference are at different scales The key insights identified are that VAE and U-Net features used in diffusion-based enhancement are sensitive to scale, causing attention to retrieve incorrect correspondences Furthermore the VAE encoder is not scale-equivariant, which prevents correction in latent space and necessitates image-space solutions

How the Training Signal is Built

MACRO resolves the scale mismatch by leveraging known 3D structure to decompose the close-up into depth planes The process involves three main components:

  1. Multi-plane depth decomposition: The close-up rendering is decomposed into P depth planes using k-means clustering on the rendered depth map Dcu to group pixels at similar depths

  2. Scale-matched reference cropping: For each plane p we compute a scale-matched crop of each reference image that shows the corresponding scene content at the same scale as it appears in the close-up view This involves calculating a scale factor sp and cropping the reference image to cover a region of 1/sp of its field of view before resizing it

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Scale-matched reference cropping will be performed by computing a scale factor sp based on object depths in both cameras, and then cropping and resizing reference images to match this scale before VAE encoding, which allows the attention mechanism to establish correct correspondences.

  2. A depth-aware attention mask will be constructed where each close-up token attends only to reference tokens from the corresponding scale-matched patch for that plane's scale, as described by Equation (6).

  3. The improved AI system can perform high-quality close-up novel view synthesis from 3D Gaussian Splatting without retraining or architectural changes by leveraging known 3D structure to decompose the close-up into depth planes and applying a training-free preprocessing and attention masking strategy.

Abstract

Close-up rendering, zooming into a scene well beyond any training camera, is important for virtual production and interactive 3D content, yet remains an open challenge. 3D Gaussian splatting (3DGS) enables high-fidelity, real-time novel view synthesis, but its rendering quality degrades at close range. Recent diffusion-based methods that enhance the rendering by conditioning on reference images from the training set produce significant artifacts in this setting. We analyze this failure and identify its root cause: the scale gap between the close-up and reference views. We show that the features in reference-conditioned enhancement models are not scale-invariant, causing cross-view attention to retrieve incorrect correspondences when the same content appears at different scales, and that this mismatch cannot be corrected in latent space because the VAE encoder is not scale-equivariant. Building on this analysis we introduce MACRO, Multi-plane Attention for Closeup Render Optimization, a training-free method for high-quality close-up novel view synthesis from 3DGS. MACRO resolves the scale gap by leveraging the scene's known 3D structure: it decomposes the close-up into depth planes, crops and resizes references in image space to match the scale of each plane before encoding, and applies a depth-aware attention mask so each token attends only to scale-matched references. The method requires no architectural changes or additional training. We further contribute two new close-up novel view synthesis benchmarks, the first standardized evaluation protocol for this setting, and demonstrate state-of-the-art results on both, outperforming existing 3DGS and diffusion-based methods on both reconstruction and perceptual metrics. Project page: https://nitzanhod.github.io/MACRO

Sources

Related papers