LLMs, You Can Evaluate It! Design of Multi-perspective Report Evaluation for Security Operation Centers
The paper designs an Analyst-wise Checklist based on SOC practitioner knowledge and creates MESSALA, a novel framework for LLMs to evaluate security reports. MESSALA uses this checklist to provide expert-level, multi-perspective feedback, proving that LLMs can quantitatively score reports and qualitatively offer actionable insights superior to existing methods.