From Transfer Probes to Expert Grouping: Evidence from LoRA Post-Training
Researchers tested if transfer probes could discover an optimal grouping of experts when a standard taxonomy is missing. They compared three groups—the reference grouping (C0), an uninformed default (C1), and a probe-derived group (C2)—under specific training conditions. Neither candidate achieved the predicted 2 percentage point gain over C0, with all candidates performing worse than the reference grouping.