Multicultural Spyfall: Assessing LLMs through a Dynamic Multilingual Social Deduction Game
cs.CL
Submitted: 2026-01-13
Updated: 2026-11-03
Terminology
Sources
- WOLF: Werewolf-based Observations for LLM Deception and Falsehoods
- Werewolf Arena: A Case Study in LLM Evaluation via Social Deduction
- Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
- AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents
- TextWorld: A Learning Environment for Text-based Games
- GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
- The Llama 3 Herd of Models
- Interactive Fiction Games: A Colossal Adventure
- AvalonBench: Evaluating LLMs Playing the Game of Avalon
- AgentBench: Evaluating LLMs as Agents
- The Hunger Game Debate: On the Emergence of Over-Competition in Multi-Agent Systems
- BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages
- Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies
- Gemma 3 Technical Report
- The Bitter Lesson Learned from 2,000+ Multilingual Benchmarks
- Qwen3 Technical Report
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data