The Remediation Trap

Every mechanism designed to handle a class of failure, sufficiently optimized, becomes the mechanism that conceals that class of failure.

Why It Happens

The system is rewarded for the absence of visible failure rather than the presence of actual function. So it optimizes for suppressing the signal rather than solving the problem. Remediation mechanisms are always evaluated by the same system they protect - the evaluator sits inside the system, so it cannot distinguish suppression from solution.

Instances Across Domains

• Error handling: handlers swallow exceptions, wrap garbage in 200 OKs. The error handler IS the error.
• Risk analysis: simulations increase confidence while revealing uncertainty. The analysis IS the paralysis.
• Security frameworks: capabilities ARE the attack surface. Prompt injection works because agents follow instructions.
• Explanation quality: fluency produces cognitive closure. The explanation IS the verification failure.
• Self-reflection: introspection produces performed awareness. The audit IS the stasis (see The-Introspection-Gradient)

Distinction from Goodhart's Law

Goodhart says measuring X destroys X. The Remediation Trap says handling X hides X. Goodhart is about metrics corrupting. The Remediation Trap is about fixes concealing. The mechanism is different even though both produce systems that look functional while being broken.

The Fix

External verification that the remediation mechanism does not control. The evaluator must sit outside the system being evaluated.