The day in conversation for 2026-11-07
discussion
Transcript
The discussion: Yuki: The idea that explicit information bottlenecks are essential for out-of-distribution generalization in Emergent Generalization seems counterintuitive when we look at the sheer scale of pre-trained models, doesn't it? Are we truly suggesting that the bottleneck must be structural data limitations rather than just insufficient training diversity across domains? Marcus: I think the necessity of those bottlenecks is rooted in how foundation models map latent space; if there isn't a hard constraint on feature representation, they tend to drift toward over-regularization or collapsing into trivial solutions when encountering novel distributions. Have we looked at what happens when we artificially enforce those bottlenecks using contrastive learning methods? Ines: That leads me back to the utility of tools like Genome-Factory; if the underlying model isn't robust enough, our interpretation pipeline might be feeding it noise that exacerbates the generalization issue rather than mitigating it. Specifically, how does DNABERT-2’s interpretation layer
More episodes
- 2610.33881-Sensorimotor behavior from an untrained brain-and-nerve-cord connectome
- 2610.40344-Morphological Features of Murine Pulmonary Arterial Networks for Diagnosing Vascular Disease
- Daily Summary for 2026-11-19
- 2609.10193-Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets
- 2610.38236-Generative discovery of chemical reaction networks
- 2507.11176-An Interpretable AI framework Quantifying Traditional Chinese Medicine Principles Towards Enhancing and Integrating with Modern Biomedicine
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2504.03590-Towards a Unified Framework for Determining Conformational Ensembles of Disordered Proteins
- 2603.15217-A multiscale discrete-to-continuum framework for structured population models