Hierarchical Alignment: Block-Level LoRA-DPO for Preference Tuning in Large Language Models.

arXiv:2510.12044v2 Announce Type: replace-cross Abstract: Preference tuning typically distributes adapter updates across every Transformer layer. We study how the placement of a fixed LoRA budget affects Direct Preference Optimization (DPO). Hierarchical Align