From Spectra to Joint Schedules in LLM Pre-training: 3+3(+2) Scaling-Law Regimes

Name: From Spectra to Joint Schedules in LLM Pre-training: 3+3(+2) Scaling-Law Regimes
Uploaded: 2026-10-15T03:48:02.554431+00:00
Duration: 248 s
Description: This research investigates how noise and learning schedules jointly shape power-law loss curves in LLM pre-training. It models loss as a dynamical response involving spectral structure and schedule parameters, leading to a 3+3(+2) classification of scaling regimes. The findings allow for predicting how different training schedules affect model performance and resource optimization.

This research investigates how noise and learning schedules jointly shape power-law loss curves in LLM pre-training. It models loss as a dynamical response involving spectral structure and schedule parameters, leading to a 3+3(+2) classification of scaling regimes. The findings allow for predicting how different training schedules affect model performance and resource optimization.

Open the full episode