How Should I Pick a Foundation Model for My Robot? In Favor of a Community Evaluation Framework for Social Robots
Eric Nichols, Alva Markelius, Hatice Gunes
cs.RO, cs.CL, cs.HC
Submitted: 2026-08-07
Updated: 2026-08-10
Comments: 5 pages, 1 figure, 1 table. Accepted at the FoRMA workshop (Foundation Models in the RO-MAN Age: Responsible Development for Social Robotics) at IEEE RO-MAN 2026, Kitakyushu, Japan. Workshop homepage: https://sites.google.com/cam.ac.uk/forma/
Project page: https://sites.google.com/cam.ac.uk/forma
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
- LiveBench: A Challenging, Contamination-Limited LLM Benchmark
- RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies
- Holistic Evaluation of Language Models
- Measuring Massive Multitask Language Understanding
- Instruction-Following Evaluation for Large Language Models
- OR-Bench: An Over-Refusal Benchmark for Large Language Models
- A Survey on LLM-as-a-Judge
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation
- LLM Evaluators Recognize and Favor Their Own Generations
- BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies