Beyond a Single Judge: The Evidence-Grounded, Social-Weighted Persona Panel for Generative UI Evaluation
Zheng Wu, Yibo Luo, Pu Zhang, Cheng Yang, Zhuosheng Zhang
cs.CL
Submitted: 2026-07-30
Code: https://github.com/Wuzheng02/ESPP
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generative UI (GenUI) lets large language models synthesize a complete, renderable interface directly from a natural-language instruction, but evaluating the quality of what they generate remains an
Terminology
Abstract
Generative UI (GenUI) lets large language models synthesize a complete, renderable interface directly from a natural-language instruction, but evaluating the quality of what they generate remains an open problem. Human evaluation is costly and rater-variant, while LLM-as-a-judge is scalable but reflects only a single implicit viewpoint, unable to capture how different populations of real users actually perceive the same interface. We propose the Evidence-Grounded, Social-Weighted Persona Panel (ESPP), a three-stage GenUI evaluation method in which a panel of psychologically diverse, evidence-grounded personas independently rates a screenshot, exchanges opinions under a trait-derived, semantically-gated bounded-confidence mechanism, and is aggregated via Delphi-inspired social weighting into a single judgment. ESPP tracks human judgment substantially more closely than a naive single-pass judge, raising Pearson r from 0.716 to 0.922, and a prompt-ensemble control recovers only about a third of this gap, isolating genuine persona and evidence grounding as the dominant source of improvement. Beyond this fidelity gain, retaining each panelist's individual rating further reveals that user subgroups agree on overall model rankings yet diverge sharply on specific rating dimensions, a structural disagreement a single homogeneous judge would systematically erase. The codes are available at https://github.com/Wuzheng02/ESPP.
Sources
- The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Generative UI: LLMs are Effective UI Generators
- LLM Critics Help Catch LLM Bugs
- Verbosity Bias in Preference Labeling by Large Language Models
- Personality Traits in Large Language Models
- Kimi K2: Open Agentic Intelligence
- Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- UI2Code^N: UI-to-Code Generation as Interactive Visual Optimization
- ReAct: Synergizing Reasoning and Acting in Language Models
- GLM-5: from Vibe Coding to Agentic Engineering
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering