Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization.

arXiv:2610.12855v1 Announce Type: new Abstract: On-Policy Self-distillation is a self-improvement paradigm that enables large language models (LLMs) to improve their capabilities by leveraging the same model as both teacher and student under different conditio