Layerwise Repacking: Variable-Length Hybrid-Attention Training without an Extra Redistribution Pass.

arXiv:2610.35989v1 Announce Type: cross Abstract: Variable-length training creates conflicting parallelism requirements across the layers of a hybrid-attention model. The longest samples require wide sequence-parallel (SP) groups of 16 or 32 ranks for full att