From Prompting to Composing: A Spatial Canvas Interface for Poster Generation

arXiv:2610.12230 · cs.CV, cs.AI · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "From Prompting to Composing".

Jane: The gist: Compo introduces a Spatial Canvas Interface that allows users to compose generation intent in two-dimensional space through four complementary binding types—semantic, identity, text,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: And what they’ve done with that data is use it to train a model called Compo, which was adapted from a pretrained image editing model. This allows it to handle these spatial inputs and those text specifications.

Jane: It supports two ways of using the model: direct inference, where you manually build the canvas yourself, and agentic mode. The agentic mode takes a high-level request and automatically plans that whole Spatial Canvas for you.

Lu: The training pipeline is quite complex; it involves a two-stage process called Poster Creation and Annotation Generation. In the first stage, they use GPT-five point five to create content and style repositories, which are then turned into prompts rendered by multiple text-to-image models for diversity <ref:2610.12230#pg1>.

Meng: Then in the second stage, they use SAM-three to segment parts of those posters, FLUX <ref:2610.12230#pg1>.two-klein-9B for relighting if needed, and Chandra-OCR-two to read the text and find where it is located spatially. That’s how they build that Compo-200K dataset.

Lalam: So the core idea is that they are using this structured way of inputting data to teach a multimodal diffusion transformer, which they adapt with LoRA, to understand the Spatial Canvas and those specifications without having to train a whole new poster generator from scratch.

Tom: That adaptation method is interesting because it keeps the generative power of the pretrained model while teaching it this new spatial composition understanding. It’s not training something totally new; it’s fine-tuning an existing structure.

The paper's summary: Jane: The authors focus on a few key areas for improvement. First, they introduce a benchmark specifically to evaluate how well the model adheres to individual binding types and also how well those bindings work together in combination.

Lu: They found that Compo achieves stronger compositional controllability when compared against both general purpose image generation models and dedicated poster generation systems, while still keeping a high visual quality.

Meng: That’s important because it means the model isn't just making pretty pictures; it’s respecting the spatial instructions you gave it in a structured way. It shows better control over the layout itself.

Tom: The paper also talks about using Reinforcement Learning, specifically DiffusionNFT, to refine the supervised model further. They optimize a trainable LoRA module with this RL post-training using three specific rewards.

Lalam: Those rewards are text reward, which checks rendering quality using sentence accuracy and normalized edit distance, instance similarity reward which checks consistency using DINO-I or CLIP-I similarities, and layout reward which measures the intersection over union between the intended and generated regions.

Jane: That multi-reward system is designed to give them complementary supervision during the RL phase, making sure it handles all those different constraints—semantic, identity, and textual—at once.

Tom: And they even gave us a precise way to tune that text reward using a formula like Rtext equals wRSen.Acc. plus one minus w RNED, letting you balance character accuracy against the edit distance feedback for poster correctness.

The paper's improvements: Jane: So, to wrap up, this Spatial Canvas Interface moves poster generation away from just typing long prompts toward explicitly composing the final poster structure using those four binding types and detailed specifications.

Tom: The main implication is that you get much stronger compositional control over the output than you would with current general image generation tools or systems like CreatiDesign, while still maintaining high visual quality.

Lu: The work shows that this structured approach to inputting data and training the model allows for efficient adaptation without needing to train a specialized poster generator from scratch.

Meng: For practical application, the agentic mode is key; it lets you start with just a high-level request and let the system plan out all the binding selections and layout details on its own before generating anything.

Lalam: Overall, "From Prompting to Composing: A Spatial Canvas Interface for Poster Generation" shows how structured input specification can lead to models that have much better compositional control over complex visual tasks.

Tom: That’s what we’ve got today on the paper, shifting the focus from just prompting to actively composing the final visual output.

Conclusion: Tom: So, to wrap up, this paper introduces "From Prompting to Composing: A Spatial Canvas Interface for Poster Generation," which is basically letting you build posters in a two-dimensional space instead of just typing text prompts.

Jane: Exactly. They’re giving users direct control by letting them use four different ways to bind things—semantic, identity, text, and pixel—all organized on a canvas with specific instructions for each part.

Lu: It’s interesting how they tackle the training side too. They built this whole supervision pipeline automatically to get that Compo-200K dataset without having to manually design every single poster pair.

Meng: That makes sense, because getting that kind of data is usually a huge bottleneck for training generative models. It lets them adapt the model efficiently using LoRA without starting from absolute zero on a poster generator.

Lalam: From my side, I see how this structured input helps me process things better; having explicit spatial and textual descriptions means I can follow the generation intent much more precisely.

Tom: And that's what they achieve with the Reinforcement Learning part, too. They use a three-part reward system to fine-tune it, focusing on text accuracy, visual consistency using DINO and CLIP similarities, and layout adherence.

Jane: So instead of just hoping for a good picture, the model is being explicitly trained to respect those different types of instructions simultaneously across the whole canvas.

Lu: It shows that when you break down generation into these specific binding constraints rather than one long text sequence, you get much better compositional control over the final output.

Meng: Practically speaking, this means if you need a poster with a specific element in a precise location and identity preserved, Compo-LongCat or Compo-Qwen might give you the fidelity needed for professional use.

Lalam: I think it’s about making the generation process more reliable because when you know exactly where things are supposed to go, it reduces the need for endless re-prompting.

Tom: It definitely shifts poster generation from a guessing game based on wording to a structured composition task using that Spatial Canvas Interface.

Jane: That's the main shift—moving toward explicit instruction rather than just descriptive prompting.

Lu: Anyway, this framework is really powerful because it shows how multimodal models can be adapted to handle these kinds of spatial reasoning tasks effectively.

Meng: It’s a solid piece of engineering because they managed to integrate that complex training process into the existing MMDiT architecture using LoRA.

Lalam: I'm looking forward to seeing how this structured input idea shows up in future multimodal architectures, especially when we look at things like image classification performance.

Yitong Wang, Fangyun Wei, Jinjing Zhao, Sirui Zhang

Fudan University · Microsoft Research

cs.CV, cs.AI

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: Project page: https://snowflakewang.github.io/Compo-Page/ GitHub: https://github.com/snowflakewang/Compo

Project page: https://snowflakewang.github.io/Compo-Page

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The gist: Compo introduces a Spatial Canvas Interface that allows users to compose generation intent in two-dimensional space through four complementary binding types—semantic, identity, text, and

Key concepts

Spatial Canvas Interface
This is the core innovation that lets users build a poster design in a 2D space rather than just typing text. It allows users to place elements precisely using four binding types: semantic (what content goes where), identity (preserving an image's look), text (placing specific words), and pixel (anchoring exact visual areas).
Binding Types
These are the four complementary ways a user can link their intent to the visual generation process. Semantic binding specifies content for a region, Identity binding keeps a reference image's style, Text binding places text exactly where desired, and Pixel binding locks down specific visual parts of an image.
Text Specifications
These are detailed descriptions used to define the appearance of posters. They cover both individual elements—like font style or color for a specific piece of text—and global appearance details, such as the overall background and artistic style that should be applied to the entire composition.

Terminology

Summary

The gist: Compo introduces a Spatial Canvas Interface that allows users to compose generation intent in two-dimensional space through four complementary binding types—semantic, identity, text, and pixel—together with Text Specifications for individual elements and global appearance.

How it works

The core innovation is the Spatial Canvas Interface which decouples intent specification from visual generation by allowing users to directly compose generation intent in two-dimensional space through four complementary forms of binding: semantic, identity, text, and pixel, together with Text Specifications for individual elements and global descriptions such as background and style.

The interface supports four complementary forms of binding: Semantic Binding specifies what visual content should be generated in a designated region; Identity Binding associates a reference image with a target region to preserve its visual identity; Text Binding places specified text at a desired location; and Pixel Binding directly anchors visual content that should be preserved.

Compo is developed as a poster generation model adapted from a pretrained image editing model to understand Spatial Canvas inputs and Text Specifications, supporting both direct inference where users explicitly construct the canvas, and agentic mode where a high-level request is automatically translated into a planned Spatial Canvas.

Training Data Generation

To train Compo, researchers developed a scalable pipeline that automatically constructs supervision for different binding types and their combinations, resulting in Compo-200K.

This pipeline consists of two stages: Poster Creation and Annotation Generation. In the Poster Creation stage, GPT-5.5 is used to construct two text asset repositories—a content repository containing diverse poster themes and their associated items, and a style repository covering a wide range of visual styles—which are then organized into a structured specification.

The resulting specification is rewritten into a natural poster generation prompt while preserving all sampled and planned information, which is then rendered with multiple text-to-image models to enhance diversity in the training data.

In the Annotation Generation stage, SAM-3 is used to segment relevant instances from the generated poster, FLUX.2-klein-9B is employed for relighting or reposing when needed, and Chandra-OCR-2 is used to recognize rendered text and recover its spatial location.

All elements are then directly rendered onto the Spatial Canvas as visual conditions, while their detailed descriptions are systematically documented in the Text Specifications, resulting in a dataset of roughly 200K samples denoted as Compo-200K.

Model Architecture and Training

Compo is built upon a pretrained multimodal diffusion transformer (MMDiT) which is adapted using LoRA to efficiently learn the proposed Spatial Canvas Interface while retaining the generative capabilities of the pretrained model.

Stage-1 involves supervised training where the Spatial Canvas and Text Specifications are jointly encoded with the target poster to adapt a pretrained MMDiT through LoRA, where both models use a frozen Qwen2.5-VL to encode multimodal specifications into visual-language tokens (VL tokens). The text branch of MMDiT processes the VL tokens, while the image branch processes the noisy target tokens together with canvas tokens.

Stage-2 involves Reinforcement Learning (RL) using DiffusionNFT to further refine the supervised model by optimizing a trainable LoRA module with a multi-reward objective tailored to the binding requirements of Compo.

The RL reward comprises three complementary components: the text reward, which evaluates rendering quality using sentence accuracy and normalized edit distance (NED); the instance similarity reward, which evaluates consistency using DINO-I, CLIP-I, and CLIP-T similarities; and the layout reward, which measures IoU between intended and generated regions.

Inference Modes

Compo supports two complementary inference modes: Direct Inference and Agentic Mode.

Direct Inference allows users to manually specify the Spatial Canvas and Text Specifications for fine-grained control over element locations, binding types, visual references, and generation requirements.

Agentic Mode enables users to generate a poster directly from a high-level natural-language request. The agent performs element extraction, layout planning, binding selection, and image search to construct a planned Spatial Canvas and Text Specifications before passing them to Compo for final poster generation.

Evaluation and Results

The benchmark evaluates adherence to individual binding types and their joint composition rather than just overall image quality.

Experiments show that Compo achieves stronger compositional controllability than both general-purpose image generation models and dedicated poster generation systems while maintaining high visual quality.

Quantitative results in Table 5 compare Compo models before and after RL post-training, showing consistent improvements across most evaluation metrics for both model variants.

Qualitative results in Figure 6 qualitatively compare Compo with representative baselines, demonstrating robust generalization to unseen themes and visual elements.

The ablation study on RL reward components shows that the three reward components provide complementary supervision during RL post-training, confirming their necessity for optimal performance.

Overall, Compo shifts poster generation from prompting toward composing by providing an explicit and structured specification of the target poster through a Spatial Canvas and Text Specifications.

The final model variants, Compo-LongCat and Compo-Qwen, outperform dedicated poster generation methods like PosterMaker and CreatiDesign across nearly all metrics, demonstrating superior compositional control.

Improvements for AI systems

  1. textbfDirect Composition Intent Specification via Spatial Canvas Interface: A system can move beyond text prompting by allowing users to directly compose generation intent in two-dimensional space. This enables explicit control by supporting four complementary binding types—semantic, identity, text, and pixel—allowing users to structure the target poster spatially rather than relying on a single one-dimensional sequence of words.

  2. textbfStructured Data Generation for Training: The system can be trained efficiently by developing a scalable pipeline that automatically constructs supervision for different binding types and their combinations, resulting in the Compo-200K dataset. This avoids the prohibitive cost of manually constructing large-scale canvas–poster pairs, enabling adaptation without training a specialized poster generator from scratch.

  3. textbfAgentic Mode for High-Level Request Translation: The system can operate in an agentic mode, where a high-level request is automatically translated into a planned Spatial Canvas. This allows users to provide only a natural language request, and the agent automatically performs element extraction, layout planning, binding selection, generating the structured representation needed for generation.

  4. textbfBinding-Aware Reinforcement Learning: The system can achieve stronger compositional controllability by using a binding-aware multi-reward objective during RL post-training. This objective optimizes three complementary rewards—text reward, instance similarity reward, and layout reward—to ensure fidelity across semantic, identity, and textual constraints.

  5. textbfPrecise Control Over Text Fidelity: The system can enforce strict text correctness by defining a combined text reward as Rtext = wRSen.Acc.+ (1−w)RNED, where the weight 'w' allows tuning between sentence accuracy and normalized edit distance to balance dense character-level feedback with poster-level correctness.

  6. textbfRobust Evaluation Protocol: The system can be evaluated using a model-agnostic benchmark that measures adherence to each binding type individually and their joint composition, rather than just overall image quality. This protocol uses specialized metrics like CLIP-T for semantic alignment and IoU for layout adherence to rigorously assess controllability.

Sources

Related papers