Feedback Forensics: A Toolkit to Measure AI Personality
cs.CL, cs.AI
Submitted: 2025-09-30
Updated: 2026-11-15
Code: https://github.com/rdnfn/feedback-forensics
Terminology
Sources
- Measuring Massive Multitask Language Understanding
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
- Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback
- The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
- VisionArena: 230K Real World User-VLM Conversations with Preference Labels
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models
- Idiosyncrasies in Large Language Models
- Evaluating Large Language Models with Psychometrics
- WildChat: 1M ChatGPT Interaction Logs in the Wild
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data