The Optimal Decoder is Tilted Temperature
The research explores optimizing Large Language Model decoding by finding a 'tilted temperature' distribution instead of a fixed temperature. It solves an optimization problem balancing output diversity (entropy floor) and plausibility (cross-entropy ceiling). The result is a dynamic sampling rule that uses token-level potential to guide the selection of optimal sampling parameters, leading to better performance in complex tasks.