Automation Bias
The human tendency to over-trust automated output and approve it with less scrutiny than a person would apply to a colleague's work — the failure mode that quietly turns Human-in-the-Loop (HITL) oversight into theater. A reviewer rubber-stamping hundreds of confident-looking agent actions per hour is laundering decisions, not overseeing them. EU AI Act Article 14 names automation bias explicitly: overseers of high-risk systems must be enabled to *remain aware* of the tendency to automatically rely on system output.
Written by Max Zeshut
Founder at Agentmelt · Last updated Sep 9, 2026
Example
A clinician using an AI documentation agent starts approving its summaries without reading them because the first fifty were accurate. The fifty-first silently drops a medication allergy. The human gate existed on paper but provided legal cover, not real oversight — the classic automation-bias trap.
Frequently asked questions
- How do you design against automation bias?
- Surface the agent's *reasoning* and *what it's unsure about*, not just its conclusion, so the human has something to actually evaluate. Route only genuinely uncertain or high-consequence items to review (so every item in the queue deserves a decision), set realistic approval SLAs, and monitor the approval rate — a 100% approval rate is a red flag that reviewers are rubber-stamping. Periodically inject known-bad items to test whether reviewers are catching them.