Autopilot flies the plane, yet we still pay two trained pilots to sit in the cockpit. So surely every AI needs a trained human watching over it? It is the most quoted argument in AI governance, and after twenty years in aerospace, including at Airbus, I can tell you it is wrong, or at least badly misapplied.
The analogy travels well because it borrows a century of aviation credibility in a single sentence. Autopilot handles the cruise, a human stays ready to take over, nobody dies. Case closed. But it smuggles in an assumption that almost never holds for a business AI system: that the cost of a mistake is catastrophic.
Look at why the pilot is really there
The pilot is not in that seat because flying is complicated. The pilot is there because of the failure mode. If the autopilot gives up over the Atlantic at 38,000 feet, the outcome is 200 deaths and hundreds of millions in insurance payouts. At those stakes, paying two highly trained humans to sit there for eight hours is simply rational economics.
Now run the same test on your own use case. If a triage workflow misroutes a support email, what actually happens? Someone spots it, reroutes it, the model gets corrected, and the world carries on. On a support system we deployed recently, that is precisely why we could push triage accuracy above 90% and let it run: the cost of the rare miss is a few minutes, not a fatality.
The failure-mode test
Before you wrap an AI tool in committees and governance frameworks, ask two questions:
- How bad is the worst realistic mistake? A misrouted ticket, or a downed aircraft?
- How quickly is that mistake caught and reversed? Minutes, or never?
When the failure mode is low and the value is high, heavy oversight is not prudence. It is a tax on your own productivity. That level of caution made sense for the cockpit. Applied to a quotation assistant or a sales tool, it is a heroic rearguard action against getting anything done.
This is how every technology has spread
When the downside is small and the upside is real, humans have always adopted fast. Cars, satnav, power tools, none of them arrived wrapped in a six-month training programme before anyone was allowed to touch them. That is not recklessness. It is how useful technology has always diffused.
The mistake is treating “human in the loop” as a moral principle rather than an economic one. Keep a human in the loop where the loop is expensive to get wrong. Everywhere else, that person is better spent on the work only they can do.
Where this shows up in practice
This is the logic behind how we build. Our service and support systems draft and triage automatically, with a human approving anything that reaches a customer, oversight sized to the actual risk rather than the imagined one. The same thinking runs through Cortex, our support intelligence engine: it handles the routine work it can ground in a source, and flags what it cannot.
So here is the question worth sitting with. When you actually map the failure mode of the AI project stuck in committee, is it anywhere near as frightening as the committee assumed?
Erwan Lhermitte spent twenty years in aerospace, defence and scientific engineering before founding Ascentis AI to build self-hosted AI for engineering-led businesses.