We are currently witnessing a real-time experiment in substituting human judgment with crude automation. Recent revelations regarding the Department of Government Efficiency (DOGE) illustrate how the “efficiency” of AI can be weaponized to bypass due process. By using AI models to mass-cancel thousands of federal grants based on simple keyword searches, the system stripped away all meaningful context in favor of speed.
From a Human Factors perspective, this is an abuse of the human-agent interdependence these systems rely on for sound reasoning (Joint Cognitive System (JCS), Hollnagel & Woods, 2005). By deploying AI to flag and terminate grants without transparent reasoning, DOGE created a system that attributes failure to the user, even if that failure was driven by system design (Moral Crumple Zone, Elish, 2019). The staff “reviewed” the lists, but because the AI provided no Uncertainty Metadata or contextual logic, there was no Common Ground (Klein et al., 2005)—the essential shared understanding between human and machine. Without knowing why the AI flagged a specific study, the human cannot effectively audit the decision.
Critically, while human oversight was technically present, the staffing of the agency with personnel who lacked deep domain expertise created a mismatched mental model. These “reviewers” were unable to recognize when the AI was hallucinating political bias into historical research. The system was not built to allow for sound review or intervention; it was built to facilitate compliance, not critical thinking.
Engineering Calibrated Trust
The path to preventing catastrophic AI failure is not found in more data or quicker results; it is found in the engineering of calibrated trust. To understand why calibration—matching a user’s trust to the system’s actual capability—is critical, we must look to the elevator. Elisha Otis didn’t actually invent the elevator; he spectacularly invented Trust in the elevator.
At the 1854 World’s Fair, Otis stood on a platform, hoisted it high above a gasping crowd, and ordered the hoisting cable to be cut. When the platform held fast, it wasn’t because of a “friendly” interface or a marketing campaign—it was because a Hard Constraint, a mechanical safety brake, had snapped into place. Otis proved that trust is not a feeling we coax from a user; it is a structural guarantee. It is the result of making the system’s safety limits observable and its fail-safes absolute.
We must apply this same rigor to AI. Authentic trust cannot be “skinned” onto an interface with better icons or conversational filler that treats computers as social actors. Using “friendliness” to mask a lack of transparency is a form of deceptive UX that leads to overtrust and, eventually, disaster.
The Shift to Coactive Design
Instead, we build trust by acknowledging that certain tasks—by their very nature—require human authority. The system must be designed to “recognize” task severity and pull the user into the process, not as an afterthought, but as a primary component. As we saw with the DOGE example, even when a human is “in the loop,” the system can still be weaponized if the UI is designed to favor speed over accuracy.
To prevent this, we must shift our focus to the Level of Automation (LoA). By realizing that AI and humans form a joint cognitive system, we can build in hard constraints—digital “safety brakes”—that prevent the system from neglecting the user for the sake of a “quick” result.
Any system that treats human oversight as a bottleneck to be bypassed is not just poorly designed; it is inherently malicious by architecture. If a system doesn’t allow for Observability and Directability, it should never have been deployed in the first place.
Building for Agency, Not Just Efficiency
True efficiency in the age of AI isn’t about how many actions we can automate in an hour; it’s about how many correct decisions we can make with the help of a machine.
The “efficiency” promised by automated systems like DOGE and others (and there are others and will be others) is often a debt that will be paid later in the form of errors, lawsuits, and lost trust. True efficiency in the age of AI isn’t about how many actions we can automate in an hour; it’s about how many correct decisions we can make with the help of a machine.
We must stop designing AI as a replacement for human expertise and start designing it as a support for it. If we continue to build systems that prioritize speed over common ground, we aren’t building progress—we are building a machine that we can no longer steer.