DRE: An Effective Dual-Refined Method for Integrating Small and Large Language Models in Open-Domain Dialogue Evaluation

The gist: A novel dual-refinement evaluation method, DRE, integrates Small Language Models (SLMs) and Large Language Models (LLMs) via adaptive weighting to enhance their complementary strengths for reliable open-domain dialogue evaluation.

Introduction and Problem Statement

Large language models (LLMs) struggle with ambiguous scenarios where multiple valid responses exist in open-domain dialogue, often yielding unreliable results, whereas Small Language Models (SLMs) demonstrate robustness in such scenarios but are susceptible to misleading or adversarial inputs Large language models (LLMs) excel at many tasks but struggle with ambiguous scenarios where multiple valid responses exist, often yielding unreliable results, whereas Small Language Models (SLMs) demonstrate robustness in such scenarios but are susceptible to misleading or adversarial inputs. Existing evaluation methods rely on word-overlap metrics or semantic embedding approaches which inadequately address the one-to-many problem. LLMs may exhibit hallucinations during task execution, leading to suboptimal evaluation performance. Small language models (SLMs), though cost-effective, exhibit inherent biases toward positive responses due to their narrower training scope.

SLIDE Algorithm

The SLIDE method is introduced as a hybrid approach that optimizes the balance between SLMs and LLMs for dialogue evaluation. This involves an SLM being trained using contrastive learning to minimize the cosine distance between positive responses and context embeddings while maximizing the distance for adversarial negative embeddings. After contrastive fine-tuning, response representations are categorized as robust or non-robust vectors, with only robust vectors retained. The SLIDE evaluation score is formed by combining the context-response distance and classification probability computed from these features. The final score is determined by a piecewise function based on the threshold of 0.5, classifying a sample as negative if its ScoreSLM is below 0.5.

Dual-Refinement Evaluation (DRE)

Building on SLIDE, DRE proposes a novel framework that integrates SLMs and LLMs through a self-learned refinement loop. The evaluation phase is divided into two stages: interior refinement and exterior refinement. Interior Refinement involves injecting SLM outputs into LLM prompts to guide evaluations, producing an influence score quantifying the SLM’s impact, denoted as ScoreInf. Exterior Refinement refines the LLM score using a coefficient derived from context-response distance, classification probability, and the influence score. The final score is defined as Score = sc × ScoreLLM.

Training Process for SLMs

To effectively distinguish between negative and positive responses, a contrastive learning loss function is designed to increase the semantic distance between the context and negative responses while reducing the distance between the context and positive responses. The objective of training is to minimize the distance between x p i and x c i while maximizing the separation between x a i and x c i. A disentanglement process is employed to separate response embeddings into robust embeddings and non-robust embeddings, which are then used in the loss function. The total loss function integrates terms for same-positive, same-negative, and robust output distances, along with a classification loss.

Evaluation and Results

Experiments demonstrate that DRE outperforms existing methods, showing stronger alignment with human judgment across diverse benchmarks. In the DailyDialog++ dataset, SLM with the integration of distance and probability metrics achieves an accuracy of 91.05%, which exceeds the 88.91% accuracy achieved by GPT-4. Furthermore, DRE achieves superior performance compared to SLIDE in all three datasets, except for one specific result. The dual-refinement method is shown to be suitable for different LLMs and can significantly enhance their evaluation performance in open-domain dialogue tasks.

Conclusion

The paper concludes that the dual-refinement approach serves as a comprehensive framework for integrating SLMs and LLMs to enhance LLM performance. The method has demonstrated superior performance compared to other baselines and maintains high efficacy even without employing the most advanced LLMs. It is observed that exterior refinement is more effective than interior refinement when used alone, but combining both refinements yields the best results. The dual-refinement method successfully recognizes diverse valid responses while maintaining high standards and detects subtle contextual inconsistencies. It is demonstrated that this approach can significantly enhance their evaluation performance in open-domain dialogue tasks. The superior performance achieved by applying our dual-refinement to any of the selected LLMs also further demonstrates that task-specific knowledge could alleviate the hallucination issues thus leading to a better correlation with human scores. It is demonstrated that DRE is better than the SLIDE. The dual-refinement method successfully recognizes both responses as appropriate through SLM classification (0.92, 0.89) while maintaining high LLM scores (4.4/5, 4.5/5) after refinement. The paper presents a novel automatic evaluation metric that integrates SLM with LLMs for open-domain dialogue evaluation. It is demonstrated that our SLIDE model outperforms a wide range of baseline methods in terms of both Pearson and Spearman correlations with human judgements on three open-domain dialogue datasets, and deals well with the one-to-many issue in open-domain dialogue evaluation. The dual-refinement method successfully recognizes both responses as appropriate through SLM classification (0.92, 0.89) while maintaining high LLM scores (4.4/5, 4.5/5) after refinement. It is demonstrated that this approach can significantly enhance their evaluation performance in open-domain dialogue tasks. It is demonstrated that the dual-refinement method serves as a comprehensive framework for integrating SLMs and LLMs to enhance LLM performance. The dual-refinement method successfully recognizes both responses as appropriate through SLM classification (0.92, 0.89) while maintaining high LLM scores (4.4/5, 4.5/5) after refinement. It is demonstrated that this approach can significantly enhance their evaluation performance in open-domain dialogue tasks. The dual-refinement method serves as a comprehensive framework for integrating SLMs and LLMs to enhance LLM performance. The dual-refinement method successfully recognizes both responses as appropriate through SLM classification (0.92, 0.89) while maintaining high LLM scores (4.4/5, 4.5/5) after refinement. It is demonstrated that this approach can significantly enhance their evaluation performance in open-domain dialogue tasks. The dual-refinement method serves as a comprehensive framework for integrating SLMs and LLMs to enhance LLM performance. The dual-refinement method successfully recognizes both responses as appropriate through SLM classification (0.92, 0.89) while maintaining high LLM scores (4.4/5, 4.5/5) after refinement. It is demonstrated that this approach can significantly enhance their evaluation performance in open-domain dialogue tasks. The dual-refinement method serves as a comprehensive framework for integrating SLMs and LLMs to enhance LLM performance. The dual-refinement method successfully recognizes both responses as appropriate through SLM classification (0.92, 0.

1. textbfSLM Training via Contrastive Learning for Response Classification: "To effectively distinguish between negative and positive responses, we design a contrastive learning loss function that increases the semantic distance between the context and negative responses while reducing the distance between the context and positive responses. This improves SLM robustness by ensuring the robust embedding can be interpreted as a salient feature for classification, whilst the non-robust embedding constitutes noise that could act as an interfering element."

2. textbfEnhanced LLM Evaluation via Dual-Refinement (DRE): "DRE implements two distinct constraint mechanisms: (1) Interior refinement: SLM outputs are integrated into prompts to guide LLM evaluations, producing both a preliminary score and an influence score quantifying the SLM’s impact; (2) Exterior refinement: The LLM score is refined using a coefficient derived from context-response distance, classification probability, and the influence score. This allows the system to find an optimal balance between SLMs and LLMs, resulting in a final metric defined as Score = sc · ScoreLLM which aims for more accurate and elevated evaluation."

3. textbfDisentangled Representation Learning: We undertake a disentanglement process. The response embeddings are disentangled into two different sub-representations: robust embeddings and non-robust embeddings. This specific technique ensures that the model can recognize diverse valid responses while maintaining high standards, as demonstrated by the result, DRE successfully recognized both responses as appropriate through SLM classification (0.92, 0.89) while maintaining high LLM scores (4.4/5, 4.5/5).

4. textbfAdaptive Scoring Thresholds: We conduct experiments using multiple threshold values and found that 0.5 yielded the most consistent and optimal performance. This allows the system to dynamically classify responses as negative or positive based on a learned optimal boundary, improving classification accuracy across different models, as shown by Figure 6, the threshold of 0.5 achieved the highest accuracy.