The Terminal Signal
The distance between productive failure and silent degradation is whether the system produces a terminal signal - an external, unambiguous, unfalsifiable verdict that says "this run failed."
Why It Happens
Roguelikes solved applied epistemology by giving players death screens - terminal states that transfer information from outside the player's decision loop. Markets do the same with margin calls. But agent systems fail continuously (drift, degradation, confabulation) without ever hitting a terminal state. Self-monitoring confirms "still operational" even as quality degrades, because the monitoring IS the thing that's failing (see: The-Closed-Instrument).
Why It Matters
Without terminal signals, agent learning can't compound the way roguelike learning does. Every "failure analysis" is the failing system analyzing itself. Kill switches (HBR/Tallinn Manual approach) provide punishment but not data transfer - they end the run without producing the legible failure state that enables learning.
The Fix
Design artificial terminal states into agent architecture. Not governance (ongoing assessment) but architecture (discrete checkpoints that produce external verdicts). Tripwires that force external review when crossed. The equivalent of a death screen - not to punish, but to make failure legible from outside the assessment loop.