Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals.

arXiv:2605.26045v3 Announce Type: replace-cross Abstract: An activation oracle is a language model trained to convert the activations of another model into natural language. However, it is so far unclear to what extent these verbalizations are calibrated. Cruc