StructZO: Structured Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning.

arXiv:2610.16451v1 Announce Type: cross Abstract: Fine-tuning large language models at scale is increasingly limited by the memory cost of backpropagation. Zeroth-Order (ZO) optimizers reduce this burden by relying only on forward function evaluations, but the