What Language Models Know But Don't Say: Non-Generative Prior Extraction for Generalization
The episode discusses the paper 'What Language Models Know But Don't Say,' which proposes using LLMs not as text generators but as structured knowledge bases. The method, LoID, extracts internal structural data from model logits to create reliable statistical priors. This approach significantly improves predictive modeling when dealing with out-of-distribution data.