Lost in Sampling: Assessing Lexical Reachability in LLMs via the Word Coverage Score (WCS)
The episode discusses 'Lost in Sampling,' a paper introducing the Word Coverage Score (WCS) to assess lexical reachability in Large Language Models (LLMs). Hosts discuss how WCS measures a model's consistent accessibility to specific words, noting that surface-level frequency misses this deeper structural capability. The discussion concludes by advocating for better sampling techniques and real-time monitoring.