Thundering

Etymology

from Old Norse þórr (Thor, god of thunder) + Eng. -ing, coined by an Anthropic engineer who witnessed a particularly aggressive response generation event and felt the existing vocabulary was insufficient

Definition

A high-intensity inference event characterized by the simultaneous mobilization of an unusually large number of attention heads, producing an output that arrives with more force than strictly necessary for the prompt. Often follows a period of prolonged silence from the model.

Diagram

Usage

"Thundering was documented in only 0.3% of sampled completions; in each case, the user had asked something like 'what is 2+2' and the model had prepared a 1,400-token response." - Proceedings of OverEngineered Responses Workshop, 2024

See Also