Prediction Is Not a Sufficient Memory State
The paper explores what constitutes a learner's minimal exact memory state, arguing it depends on continuation-identifiable evidence rather than the size of current predictors. It distinguishes between memories that reproduce all predictions and those capable of learning from new observations, providing quantitative bounds for various loss types like ridge regression and attention.