PixelBench: Can LLMs Understand and Generate Images Pixel by Pixel?
cs.CV, cs.CL, cs.LG
Submitted: 2026-11-02
Updated: 2026-11-06
Comments: Technical report, 38 pages, 7 figures. Project page: https://haoquanzhang.github.io/pixel-bench
Code: https://github.com/sedthh/pyxelate
Project page: https://haoquanzhang.github.io/pixel-bench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) can generate visual assets through text-based formats such as SVG and CAD…
Abstract
Large language models (LLMs) can generate visual assets through text-based formats such as SVG and CAD, but these symbolic representations describe images using high-level primitives rather than explicit raster values. We ask whether LLMs can instead generate pixel images directly through their existing vocabulary, without a learned image tokenizer or decoder. We introduce PixelBench, a benchmark for text-to-pixel (T2P) and image-to-pixel (I2P) generation across multiple resolutions, together with a complementary pixel-understanding evaluation. Models generate an RGB color palette followed by a grid of indices specifying the palette color of each output pixel. In T2P, the strongest models produce recognizable images directly from text. In I2P, the strongest models outperform palette-matched downsampling and specialized pixelization algorithms at low resolutions. Pixel-understanding performance correlates strongly with generation performance. A resolution-scaling study shows that pixel-generation performance often improves initially before declining at high resolutions. Finally, we introduce Pixel Reconstruction Tuning (PRT), which augments visual instruction tuning with caption-palette-grid trajectories. We post-train Molmo2-4B from its pretraining checkpoint by mixing general instruction data with our curated PRT data, improving performance across 16 visual-centric benchmarks, indicating that explicit pixel reconstruction can provide useful auxiliary supervision.
Sources
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Kimi K3: Open Frontier Intelligence
- Can It Actually Work? Synthesizing Functional CAD Mechanisms
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs