The Necessity of Setting Temperature in LLM-as-a-Judge
The study systematically investigated how decoding temperature affects LLM judge performance across different tasks and models. Results show that low temperatures ensure high consistency and accuracy, while high temperatures increase errors but can be useful for complex, ambiguous cases. This suggests temperature should be a task-dependent setting balancing stability against exploratory reasoning.