DebugLM: Learning Traceable Training Data Provenance for LLMs
DebugLM creates a framework where LLMs can trace their generated outputs back to specific training data sources. It works by teaching the model to associate its responses with unique provenance tags during training. This allows developers to pinpoint exactly which datasets or stages of training caused undesirable behaviors, enabling targeted fixes without retraining.