SalamahBench: Dialect and Category Level Safety Evaluation of Arabic Language Models
SalamahBench is a unified benchmark for testing Arabic Language Models (ALMs) safety across 12 MLCommons hazard categories. It combines and harmonizes existing datasets through AI filtering and human validation to create a standardized, native-language evaluation tool. Results show significant variation in model safety performance, highlighting the need for specialized safeguard systems over native ALMs alone.