Formally Verified Autoresearch for Analyzing the Expressivity of Multi-Head Attention.

arXiv:2610.25341v1 Announce Type: cross Abstract: Multi-head attention is a central component of the transformer architecture, yet how its expressivity depends on the number of heads remains poorly understood. We study a one-layer, attention-only transformer w