A failure of Imagination: VLMs fail to use visual representations for text, but self-distillation can help.

arXiv:2610.25470v1 Announce Type: new Abstract: Vision-language (VL) models are typically built by fine-tuning a language model to attend to visual features in service of a Visual Question-Answering objective. We ask whether this alignment changes how language