Koa-decision: Fast and Consistent Structured Decision Making with Generative LLMs
Koa-decision introduces a method for fast and consistent structured decision-making using generative LLMs. It reformulates classification as constrained generation with single-token outputs, employing atomic label tokens. This approach achieves frontier accuracy at sub-second speeds, making complex decision tasks practical for real-time applications.