CODEBLOCK: Learning to Supervise Code at the Right Granularity
CODEBLOCK is a framework that solves a problem where standard fine-tuning of code LLMs uses uniform loss across all tokens, which is inefficient. It proposes selecting entire, structure-complete code blocks instead of individual tokens. By combining item partitioning, utility scoring based on token logic, and data-flow analysis to prioritize important dependencies, CODEBLOCK achieves better performance than full token supervision while only using about 1.9% of the original training tokens.