Emergent Inverse-Depth Scaling From Nonlinearity In Attention.

arXiv:2610.11063v1 Announce Type: cross Abstract: Scaling laws describe power-law improvements in model performance with dataset size and parameter count, yet their underlying mechanisms are not fully understood. To explain the parameter count scaling, existin