Spectral Overfitting in Noisy Linear Probing of Pretrained Representations.

arXiv:2603.02293v3 Announce Type: replace-cross Abstract: Frozen pretrained features are often treated as a safe interface for downstream learning: only a small linear readout is trained, while the backbone is fixed. We show that this readout can still overfit