HiddenObjects: Scalable Spatial Prior Extraction for Object Placement from Diffusion Models

arXiv:2604.10675 · cs.CV · Submitted 2026-04-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "HiddenObjects: Scalable Spatial Prior Extraction for Object Placement from Diffusion Models".

Jane: Here's the gist of the paper. HiddenObjects: Scalable Diffusion-Distilled Spatial Priors for Object Placement is a method to learn explicit, class-conditioned spatial priors for object placement in natural scenes by distilling implicit knowledge from text-conditioned diffusion models <ref:2604.10675#pg2>.

Tom: This approach addresses the limitations of sparse human annotations and artifact-prone synthetic datasets by constructing a large-scale dataset and distilling the resulting priors into a lightweight model for fast inference <ref:2604.10675#pg4>.

Jane: Dataset Construction The framework constructs HiddenObjects, a large-scale dataset comprising 27M placement annotations evaluated across 27k distinct scenes with ranked bounding box insertions for different images and object categories <ref:2604.10675#pg2>.

Tom: This dataset is constructed by selecting images from Places365

63: and pairing object categories with background categories to avoid implausible insertions The final dataset consists of 27,010 positively annotated backgrounds and 2,097,587 positive bounding box annotations <ref:2604.10675#pg4>.

Paper summary: Jane: We’ve been talking about HiddenObjects: Scalable Spatial Prior Extraction for Object Placement from Diffusion Models. To wrap up, what’s the big picture here for the people listening who aren't deep into the technical details?

Tom: This paper shows a way to bypass the limitations of sparse manual annotations and artifact-prone synthetic datasets by learning explicit placement rules directly from text-conditioned diffusion models. They built twenty-seven million placements across twenty-seven thousand scenes, and distilling that knowledge into a fast model cuts computation time down by over five orders of magnitude.

Lu: The authors successfully extracted class-conditioned spatial priors that capture human-aligned placement expectations, even while acknowledging some biases inherited from the generative supervisors. This means we get data augmentation or scene understanding tools based on real human placement statistics at scale.

Meng: From a deployment view, if you can distill a prior like this into a model that runs in milliseconds and uses relatively little memory, it moves this kind of structured knowledge from being an expensive annotation task to something that can be used for real-time applications.

Jane: So, the conclusion is that HiddenObjects provides a scalable framework for extracting these spatial priors. It’s not just about getting more data; it’s about creating a lightweight model that allows us to perform fast inference and use those learned placement statistics in image editing or scene understanding tasks.

Tom: Exactly. We outperformed existing placement baselines and zero-shot vision-language models, and we notably surpassed sparse human annotations in downstream image editing quality, which is a pretty concrete win for the system's usefulness.

Conclusion: Jane: So we've seen how this paper builds HiddenObjects to extract spatial placement knowledge from diffusion models, and now we're looking at what they actually achieved with that whole process.

Tom: The title is HiddenObjects, and the authors are presenting a method for learning explicit, class-conditioned spatial priors for object placement. It sounds like they’re trying to get the AI to understand *where* things belong in a scene based on text.

Lu: Exactly. They are distilling implicit knowledge from text-conditioned diffusion models into these explicit spatial priors for object placement. The core idea is taking messy, hard-to-get human data and turning it into a lightweight model for fast inference.

Meng: I’m interested in the scale here because they built this dataset with twenty-seven million placements across twenty-seven thousand different scenes. That kind of data generation is what makes it scalable, but the real question is how much that knowledge actually transfers to something usable.

Jane: And what they found was that their pipeline generates annotations that are significantly better for image editing than anything human annotates, with an average score of three point nine zero versus two point six eight. That's a big difference.

Tom: That number is interesting, but let's look at the performance of the distilled model itself. It’s outperforming all existing placement baselines and even zero shot vision language models by a wide margin.

Lu: The distillation model achieves a latency of three point seven seven milliseconds per image when tested on one hundred images, which translates to two hundred thirty thousand times speedup compared to the full pipeline they used to create the priors. That's massive for real-time use.

Meng: three point seven seven milliseconds is fast enough for deployment, but what about the cost of that distillation process itself? How much compute did it take to get this distilled model running?

Jane: The paper says it required eighteen hundred eighty nine megabytes of peak GPU memory during testing, which is a reasonable size for an efficient model. It’s not some massive new architecture requiring hundreds of gigabytes just to run inference.

Tom: So the summary is that they took text-conditioned diffusion knowledge, built a huge dataset from it, and distilled it into a tiny model that works faster and better than previous methods, especially when it comes to editing quality.

Lu: It’s about creating a foundation for controllable image editing data augmentation or scene understanding because you have these learned placement statistics that are aligned with human expectations.

Jane: So while the results are impressive in terms of speed and quality compared to existing work, we should also be aware of their limitations, specifically how the resolution limit of the diffusion models introduces a perspective bias when generating small objects.

Meng: That's a fair point. If you try to generate really tiny things at high detail with these models, they tend to produce low-quality results. So the applicability is really tied to what kind of object placement and scene detail you are aiming for.

Technical University of Denmark · University of Copenhagen · Pioneer Centre for AI

cs.CV

Submitted: 2026-04-12

Updated: 2026-11-05

Comments: Accepted to WACV27

Project page: https://hidden-objects.github.io

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: HiddenObjects: Scalable Diffusion-Distilled Spatial Priors for Object Placement is a method to learn explicit…

Key concepts

HiddenObjects Dataset
This is a large-scale dataset containing 27 million object placement annotations across 27,010 scenes. It was built by pairing images from Places365 with different object categories to create diverse and realistic training data for the system.
Spatial Prior Extraction Framework
This process involves three steps: an inpainter synthesizes potential object placements, a verifier checks if the placement is correct, and a ranker scores the best options. This systematically extracts human-aligned spatial preferences from image-object pairs.
Model Distillation
A lightweight model is trained to mimic the complex spatial prior knowledge learned by diffusion models. It uses a specific loss function combining L1 distance and GIoU, weighted by reward scores, resulting in a fast model capable of real-time inference.

Terminology

Summary

HiddenObjects: Scalable Diffusion-Distilled Spatial Priors for Object Placement is a method to learn explicit, class-conditioned spatial priors for object placement in natural scenes by distilling implicit knowledge from text-conditioned diffusion models <ref:2604. p. 010675#pg2>. This approach addresses the limitations of sparse human annotations and artifact-prone synthetic datasets by constructing a large-scale dataset and distilling the resulting priors into a lightweight model for fast inference <ref:2604. p. 010675#pg4>.

Dataset Construction

The framework constructs HiddenObjects, a large-scale dataset comprising 27M placement annotations evaluated across 27k distinct scenes with ranked bounding box insertions for different images and object categories <ref:2604.10675#pg2>. This dataset is constructed by selecting images from Places365 [63] and pairing object categories with background categories to avoid implausible insertions The final dataset consists of 27,010 positively annotated backgrounds and 2,097,587 positive bounding box annotations <ref:2604. p. 610675#pg4>.

Spatial Prior Extraction Framework

The construction of a spatial prior S(x, c) between a background image x and an object c is broken down into three fundamental operations First, an inpainter I synthesizes class-consistent object insertions into a set of proposed regions. Secondly, a verifier V validates whether the inpainter correctly added the target object in the proposed region. p. 4 Thirdly, for each verified insertion, a Ranker R quantifies preference conditional to the object. p. 4 The spatial prior S(x, c) is defined as a set of (b i, r i) where r i is the score of a corresponding bounding box b i. p. 5

Pipeline Validation and Downstream Performance

The pipeline evaluation reveals that the automatically generated annotations yield significantly higher image editing quality than human annotations (3.90 vs. 2.68 average ImgEditJudge score [57]). p. 2 The distilled model outperforms all existing placement baselines and zero-shot Vision-Language Models by a large margin. p. 2 Specifically, the distilled model achieves a latency of 3. p. 277 ms/image on average tested on 100 images with 188. p. 2 p. 29 MB peak GPU memory yielding a 230,000× speedup and 300× memory reduction. p. 2

Systematic Biases Analysis

The analysis of spatial and scale distributions shows that the pipeline yields a more diffuse distribution with higher spatial variance compared to traditional object placement datasets. While the extracted priors successfully capture human-aligned placement expectations, they also inherit certain photographic composition biases from their generative supervisors. p. 2 The resolution limit of current diffusion models translates into a perspective bias where small-scale generation attempts reliably yield low-quality completions.

Model Distillation for Object Placement

The distillation model is formulated as a multi-class, closed-set prediction task using a DETR-style architecture. The loss function combines L1 distance and Generalized Intersection over Union (GIoU) explicitly weighted by the normalized reward scores. p. 5 The final objective is L = Lbbox + 0. p. 55Lplaus. This model performs best with 56. p. 96% mAP and 62.9% IoU50@1 on HiddenObjects. p. 0 The distilled model achieves a latency of 3. p. 277 ms/image on average tested on 100 images with 188.9 MB peak GPU memory. p. 2

Conclusion

We presented a scalable framework for extracting class-conditioned spatial priors from diffusion models bypassing the limitations of sparse manual annotations and artifact-prone synthetic datasets. Our pipeline produces HiddenObjects, a dataset of 27M evaluated placements across 27k backgrounds that encodes human-aligned placement statistics. p. 0 Distilling these priors into a lightweight model enables real-time inference reducing computation by over five orders of magnitude compared to the full pipeline. p. 2 We outperform both dedicated placement models and zero-shot VLMs and notably surpass sparse human annotations in downstream image editing quality. p. 1 The extracted spatial priors offer a useful foundation for controllable image editing data augmentation and scene understanding. p. 3

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Scalable Dataset Construction for Real-World Priors: The framework constructs HiddenObjects, a large-scale dataset comprising 27M placement annotations, evaluated across 27k distinct scenes, enabling learning of human-aligned placement expectations through aggregated heatmaps.

  2. Improved Object Placement Quality: The proposed pipeline yields significantly higher image editing quality than sparse human annotations (3.90 vs. 2.68 average ImgEditJudge score), demonstrating that manual placement labels are inherently ambiguous and less suitable for inpainting.

  3. Distillation for Fast Inference: The extracted priors are distilled into a lightweight model that achieves a 230,000× faster inference speed compared to the full exhaustive extraction pipeline, enabling fast object placement proposals for downstream applications.

  4. Robust Prior Extraction: The three-step framework—Inpainter, Verifier, and Ranker—uses a diffusion-based inpainting pipeline combined with an image-based reward model to construct the spatial prior S(x, c), ensuring that only plausible placements (i.e., (i)) contribute to the spatial prior.

  5. Enhanced Model Generalization: The resulting distilled model outperforms all existing placement baselines and zero-shot Vision-Language Models by a large margin, confirming that dense, rewardweighted spatial priors are essential for learning robust placement distributions.

Sources

Related papers