When automation makes us dumber

Automation severs ends from means. It makes getting what we want easier, but it distances us from the work of knowing.

- Nicholas Carr in The Glass Cage

Nicholas Carr's The Glass Cage is a thoughtful critique of automation. Carr's core argument is that, as we automate our work, we change the nature of the work itself and change ourselves in the process. These changes result in degradation of our skills and even our humanity.

This criticism isn't news to me. There's a convincing argument, recapped by Carr, that the crash of Air France 447 was enabled by autopilot systems — not failure of the systems themselves but, rather, failure of pilots to maintain their skills when autopilot systems made those skills useless most of the time.

The medical field has its own examples of this phenomenon. My dad, a doctor who has practiced for over 50 years, laments that the variety of medical tests available today has allowed doctors to fall out of practice with their physical exam skills. Instead of making quick, intelligent decisions based on examination of physical evidence, doctors add costs and delays by over-relying on batteries of tests. He once discovered a huge, palpable life threatening tumor that escaped the eye of doctors conditioned to ordering tests with minimal justification.

As an engineer, the notion that automation can degrade our abilities is both intuitive and uncomfortable. Two decades ago, as a student in MIT's 6.001, I realized that I did my programming homework faster if I wrote it on paper before attempting to run or test it on a computer. My mind helped me find answers more quickly than the computer did precisely because my brain was forced to work to solve problems on paper.

When I ran GameChanger's engineering team, we had to wrestle with whether some forms of automation were good or bad. And it wasn't obvious. I initially feared that AWS autoscaling would isolate our engineers from understanding our traffic patterns or understanding how our systems were bootstrapped. I believed that autoscaling's false sense of security slowed our reaction to production incidents, at least initially. In time, though, the team simplified how server provisioning and bootstrapping worked. Instead of trying to manage complexity with automation, our team eliminated it or moved it to a pre-launch build step (via Docker). In this simplified world, our automation worked and our availability surged to 99.99% in our peak season.

This experience helped me distill my feelings about automating processes. There's more leverage in simplification than in automation. Automation doesn't eliminate complexity. It hides it, and that's bad. I now advocate for automating that which is simple and simplifying that which is not. I resist automating whatever can't be made simple.

comments powered by Disqus