Optimal Transport Depth Up-Scaling
Scaling large language models is costly and often causes performance drops due to mismatched neuron connections between layers. Optimal Transport Depth Up-Scaling (OpT-DeUS) solves this by using Optimal Transport to align and fuse adjacent Transformer blocks when creating new layers, ensuring better function preservation during depth scaling.