Reasoning with Sampling: Cutting at Decision Points

Name: Reasoning with Sampling: Cutting at Decision Points
Uploaded: 2026-10-03T05:19:12.117753+00:00
Duration: 321 s
Description: The Entropy-Cut Metropolis–Hastings algorithm is a training-free method to sample from sharpened versions of base language models. It identifies key decision points by looking for high next-token entropy jumps and resamples there instead of uniformly. This technique improves reasoning performance on benchmarks like MATH500 without needing extra training or curated datasets.

The Entropy-Cut Metropolis–Hastings algorithm is a training-free method to sample from sharpened versions of base language models. It identifies key decision points by looking for high next-token entropy jumps and resamples there instead of uniformly. This technique improves reasoning performance on benchmarks like MATH500 without needing extra training or curated datasets.

Open the full episode