BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training

Name: BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training
Uploaded: 2026-09-06T11:47:06.007019+00:00
Duration: 440 s
Description: The episode discusses a paper called 'BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training.' The hosts explain how current LLM training methods are inefficient due to monolithic data handling. They detail how BASP solves this by organizing sequences into smaller, parallel groups, resulting in up to 1.32x speedup and maintaining model accuracy.

The episode discusses a paper called 'BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training.' The hosts explain how current LLM training methods are inefficient due to monolithic data handling. They detail how BASP solves this by organizing sequences into smaller, parallel groups, resulting in up to 1.32x speedup and maintaining model accuracy.

Open the full episode