AlienBench: A Benchmark of Structural Novelty in Language Model Outputs
cs.CL, cs.AI
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/joetm/alienbench
Terminology
Sources
- Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
- AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text
- LLM Evaluators Recognize and Favor Their Own Generations
- Recognising, Anticipating, and Mitigating LLM Pollution of Online Behavioural Research
- Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
- Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
- NoveltyBench: Evaluating Language Models for Humanlike Diversity
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data