Is Escalation Worth It? On the Depth of LLM Cascades.

arXiv:2605.06350v2 Announce Type: replace-cross Abstract: LLM cascades, in which a cheap model defers to an expensive one on low-confidence queries, are widely used to reduce inference cost. Given a pool of models, a practitioner must decide how many models to