DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures
DynaMiCS is a method for fine-tuning large language models across multiple domains by dynamically adjusting which training data to use. It treats this as an optimization problem where the goal is to maximize performance on desired tasks while strictly maintaining performance levels on other restricted tasks. It achieves this by iteratively calculating how much each dataset helps each domain and using these estimates to select the best data mixture.