WithEveryone: Unified Planning and Identity Grounding for Group Image Generation

summary

The gist

WithEveryone introduces a unified framework for generating coherent group images from five to ten reference identities by injecting each selected identity as an addressed token and predicting a…

In short

WithEveryone creates a unified framework to generate coherent group images from five to ten reference identities by treating each identity as an addressed token and predicting a structured layout plan. This method uses Layout Chain of Thought planning and Layout-Grounded ID Loss to organize spatial relationships, allowing the model to retain and distinguish multiple identities effectively without relying on unstable embedding-based face matching.

Key concepts

Layout Chain of Thought (Layout CoT)
This is a structured planning process where the model predicts a detailed layout plan before generating an image. This plan includes specific information like person indices, bounding boxes, and how each planned person relates to a reference identity. It extends standard planning techniques to ensure every generated person is correctly tied to its source identity.
Layout-Grounded ID Loss (LG-ID Loss)
This is a novel loss function that supervises the model's output by looking at the annotated layout regions rather than relying on matching faces in an embedding space. By grounding the supervision in the known spatial annotations, it successfully allows the system to scale identity retention beyond just two people.
Identity Embeddings as ID Tokens
Instead of treating reference images as simple inputs, each reference person's identity is extracted into a specific embedding that acts like an addressed token. This token is then injected into the interleaved training sequence. This forces the model to explicitly condition its generation on which specific identity it should be rendering at any given point.
Representation Forcing
This technique adapts representation forcing by requiring the model to generate a specific, predictable representation for each target person before synthesis begins. This ensures that when generating a group image, the model has already established and maintained a distinct visual identity for every individual being depicted.

Terminology used across episodes

This episode discusses

The paper

WithEveryone: Unified Planning and Identity Grounding for Group Image Generation · Read on arXiv

Fudan University · Hunyuan, Tencent Company of the University of Hong Kong

Identity-preserving image generation becomes increasingly unreliable when a scene must contain many specified people. Beyond retaining each identity, the model must bind every reference to a distinct person and location, while training-time identity losses must establish correspondence among several noisy predicted faces. We introduce WithEveryone, a unified framework for generating group images of 5-10 reference identities. WithEveryone injects each selected identity as an addressed token, predicts a structured identity-layout plan, and renders the plan as a visual condition. Its key objective, Layout-Grounded ID Loss, uses annotated face regions to supervise the intended identities directly, avoiding unstable embedding-based face matching; ID Representation Forcing additionally trains a prediction for each identity before image synthesis. WithEveryone leads in identity similarity (0.538 versus GPT-Image-2.5's 0.472), with lower Copy-Paste (0.149 versus 0.197), 94.5% identity coverage, and 9.6% identity collisions. These results show that explicit identity-layout grounding enables identity-preserving generation to scale to larger groups without relying on direct reference-face copying.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "WithEveryone: Unified Planning and Identity Grounding for Group Image Generation".

Jane: Here's the gist of the paper. WithEveryone introduces a unified framework for generating coherent group images from five to ten reference identities by injecting each selected identity as an addressed token and predicting a structured identity–layout plan <ref:2608.20336#pg2> This approach addresses the challenge of retaining and distinguishing multiple identities while organizing their spatial and bodily relationships in a single generation context <ref:2608.20336#pg6>.

Tom: Pruning redundant reasoning tokens The model employs an interleaved sequence where reference images are loaded as ID tokens, and the understanding side autoregressively predicts a structured Layout Chain of Thought (Layout CoT) containing person indices, bounding boxes, identity–layout bindings, and pose keypoints <ref:2608.20336#pg5>.

Jane: This structured planning extends ATLAS-style planning to an identity-aware setting where every planned person is tied to a reference identity <ref:2608.20336#pg6>.

Tom: A deterministic renderer then converts this plan into a visual condition that jointly conditions the flow-based image generation <ref:2608.20336#pg5>.

Title and authors: Tom: We started with WithEveryone: Unified Planning and Identity Grounding for Group Image Generation, and we talked about how it tackles the problem of generating coherent group images from five to ten identities in one go.

Jane: So, to recap what that means is that they're using an addressed token system to inject each identity into the generation process and predict a structured layout plan.

Lu: That planning step involves predicting things like person indices, bounding boxes, identity layout bindings, and pose keypoints, which extends traditional ATLAS-style planning into an identity-aware setting.

Tom: And they use a deterministic renderer to take that plan and turn it into the visual condition that conditions the flow-based image generation.

Meng: So how do they train this thing? What's the sequence of data? Is it just random images and prompts?

Jane: It’s an interleaved sequence where reference images are loaded as ID tokens, then identity selection and ID loading happen, followed by layout planning, the rendered layout condition, a summary and detailed caption.

Lalam: They also do target identity representation prediction right before the final target image generation happens to ensure things are aligned.

Tom: And to make sure those identities are recognized properly in the sequence, they extract identity embeddings for each reference person and project them into an ID token position.

Lu: That ID Representation Forcing part is key because it forces the model to predict a representation aligned with each target person before synthesis, which keeps them addressable in that shared context.

Jane: It sounds like they are really forcing the model to maintain identity alignment throughout the whole process, not just at the end.

Tom: And that leads us to how they improve it further, because they found a few specific techniques that really boost performance beyond what's already there.

Meng: What’s one of those key improvements you want to highlight right now? Something concrete we can talk about.

Jane: They focus on the Layout-Grounded ID Loss, which is crucial because it grounds the output-side identity supervision directly in the layout annotation instead of using embedding-based face matching.

Lu: That’s a smart move because it removes that common failure mode that usually stops you from applying identity losses beyond two people when you use embeddings.

Tom: It means they supervise each identity specifically within its own annotated region, which lets it scale beyond two people without the model getting unstable with those embedding matches.

Jane: So the core idea is moving away from matching faces based on embeddings and tying it directly to where the person is supposed to be in the layout.

Lalam: And I agree that this output-side supervision, as they conclude, is actually the single biggest source of identity gain in their whole system.

Tom: So, we’re moving from general embedding matching to something more spatially specific for better results on group images.

Lu: It confirms that layout-grounded output-side supervision is the main driver here for scaling up identity fidelity.

The paper's summary: Tom: We've talked about the mechanics of how they train this thing, and now let’s get to the core summary of what WithEveryone actually achieves.

Jane: So, what is it that they claim is the main outcome when you look at the overall performance metrics?

Tom: The main claim is that this unified framework successfully generates coherent group images from five to ten reference identities in one semantically and visually coherent image.

Lu: It’s about overcoming the issue where models tend to duplicate faces or merge people when you try to generate more than a couple of people, which is what they were trying to fix.

Jane: They are showing that this framework solves the problem of generating more specified people in one image rather than having to repeatedly generate independent single-person images.

Tom: They’re hitting target-context identity similarity of zero point four nine nine against zero point four six two for GPT-Image two on that benchmark, and a copy-paste score is only zero point zero five five versus the higher number on an identity-disjoint benchmark, which shows better retention of unique identities.

Meng: That sounds like a solid improvement if those scores are compared fairly against existing state of the art systems.

Jane: And they also show that when you look at how they rank across different group sizes, WithEveryone ranks first in every group size when ArcFace similarity is broken down by the number of references.

Tom: That consistency across the five-to-ten range is what’s really telling us about the stability of their approach.

Lu: It suggests that this isn't just a lucky hit for a specific number; it’s a more robust methodology across that entire range.

Jane: So, if you only listen to the show, this paper implies that we can finally get reliable group image generation from five to ten people in one coherent picture.

Tom: That’s the big takeaway for the listener: a unified framework for group image generation is now achievable without major degradation in identity similarity.

Meng: And I wonder what does this mean for us on the production side, though? Does it mean we can actually deploy something that handles more than two people reliably?

The paper's improvements: Tom: Okay, so let's drill down into the specific improvements they made and what those changes actually do to the mechanics of the system.

Jane: Can you give us a concrete example of how one of these parts improves things over just doing nothing?

Lu: The Layout Chain of Thought is crucial because it offloads that complex planning burden to the understanding side, allowing it to resolve spatial and bodily relationships among people before they even hit the rendering stage.

Tom: So instead of trying to plan all those spatial interactions at once, the model predicts a structured layout chain that details person indices, bounding boxes, identity layout bindings, and pose keypoints.

Meng: That seems like a lot of detail for the understanding side to calculate accurately on top of existing models.

Jane: But they solve that by tying every planned person directly to a reference identity in this planning process.

Lu: And then they introduce ID Representation Forcing, which requires the model to predict a representation aligned with each target person before synthesis, which ensures those identities stay addressable in the shared context.

Tom: So it’s like you’re doing two things at once: planning the layout and making sure you have identity context for every person simultaneously.

Jane: And then they add the Layout-Grounded ID Loss, which grounds that supervision in the layout annotation instead of using embedding-based face matching.

Lu: That loss removes that limitation where you could apply these losses beyond two people because it supervises each identity in its own annotated region independently.

Tom: So, instead of relying on a single global embedding match for everything, they get localized supervision for every person based on their specific layout spot.

Conclusion: Jane: So, to wrap up the paper WithEveryone: Unified Planning and Identity Grounding for Group Image Generation. What’s the final word you want to leave with us?

Tom: The main thing is that they proved that combining explicit identity supervision with structured relational planning works for multi-person generation.

Lu: It shows that plan prediction itself, rather than just plan execution is an important remaining source of error in group image generation.

Meng: So the final word from your side? What’s the biggest hurdle left for future work?

Lalam: I think getting a solid understanding of how to fix that planning prediction error is where the next big challenge lies.

Jane: So, to summarize, it’s about using ID tokens and layout planning to build a solid foundation for scaling up identity fidelity in group images.

Tom: It’s a lot of work on training, but the results show that this approach is effective for getting better group images when you scale from five to ten people.

Lu: We just need to keep pushing that plan prediction part until we get it right so we can really push the system further.

Meng: And for production, I’m still focused on making sure the deployment doesn't break at scale with all this new planning overhead.

More episodes

← Home