Wiring Beats Blending: Structure-Aware Compensation for Transformer Downscaling.

arXiv:2608.02829v5 Announce Type: replace-cross Abstract: Model families are trained size by size. Can a pretrained large model instead be converted into a smaller sibling? We characterize the 1.4B->410M conversion in Pythia end to end. Representations align s