Plato foresaw the danger of automation

Automation takes many forms and as members of a culture that reveres technology, we generally perceive automation in terms of its output: what it accomplishes, be that through manufacturing, financial transactions, flying aircraft, and so forth.

But automation doesn’t merely accomplish things for human beings; it simultaneously changes us by externalizing intelligence. The intelligence required by a person is transferred to a machine with its embedded commands, allowing the person to turn his intelligence elsewhere — or nowhere.

Automation is invariably sold on the twin claims that it offers greater efficiency, while freeing people from tedious tasks so that — at least in theory — they can give their attention to something more fulfilling.

There’s no disputing the efficiency argument — there could never have been such a thing as mass production without automation — but the promise of freedom has always been oversold. Automation has resulted in the creation of many of the most tedious, soul-destroying forms of labor in human history.

Automated systems are, however, never perfect, and when they break, they reveal the corrupting effect they have had on human intelligence — intelligence whose skilful application has atrophied through lack of use.

Nicholas Carr writes: On the evening of February 12, 2009, a Continental Connection commuter flight made its way through blustery weather between Newark, New Jersey, and Buffalo, New York. As is typical of commercial flights today, the pilots didn’t have all that much to do during the hour-long trip. The captain, Marvin Renslow, manned the controls briefly during takeoff, guiding the Bombardier Q400 turboprop into the air, then switched on the autopilot and let the software do the flying. He and his co-pilot, Rebecca Shaw, chatted — about their families, their careers, the personalities of air-traffic controllers — as the plane cruised uneventfully along its northwesterly route at 16,000 feet. The Q400 was well into its approach to the Buffalo airport, its landing gear down, its wing flaps out, when the pilot’s control yoke began to shudder noisily, a signal that the plane was losing lift and risked going into an aerodynamic stall. The autopilot disconnected, and the captain took over the controls. He reacted quickly, but he did precisely the wrong thing: he jerked back on the yoke, lifting the plane’s nose and reducing its airspeed, instead of pushing the yoke forward to gain velocity. Rather than preventing a stall, Renslow’s action caused one. The plane spun out of control, then plummeted. “We’re down,” the captain said, just before the Q400 slammed into a house in a Buffalo suburb.

The crash, which killed all 49 people on board as well as one person on the ground, should never have happened. A National Transportation Safety Board investigation concluded that the cause of the accident was pilot error. The captain’s response to the stall warning, the investigators reported, “should have been automatic, but his improper flight control inputs were inconsistent with his training” and instead revealed “startle and confusion.” An executive from the company that operated the flight, the regional carrier Colgan Air, admitted that the pilots seemed to lack “situational awareness” as the emergency unfolded.

The Buffalo crash was not an isolated incident. An eerily similar disaster, with far more casualties, occurred a few months later. On the night of May 31, an Air France Airbus A330 took off from Rio de Janeiro, bound for Paris. The jumbo jet ran into a storm over the Atlantic about three hours after takeoff. Its air-speed sensors, coated with ice, began giving faulty readings, causing the autopilot to disengage. Bewildered, the pilot flying the plane, Pierre-Cédric Bonin, yanked back on the stick. The plane rose and a stall warning sounded, but he continued to pull back heedlessly. As the plane climbed sharply, it lost velocity. The airspeed sensors began working again, providing the crew with accurate numbers. Yet Bonin continued to slow the plane. The jet stalled and began to fall. If he had simply let go of the control, the A330 would likely have righted itself. But he didn’t. The plane dropped 35,000 feet in three minutes before hitting the ocean. All 228 passengers and crew members died.

The first automatic pilot, dubbed a “metal airman” in a 1930 Popular Science article, consisted of two gyroscopes, one mounted horizontally, the other vertically, that were connected to a plane’s controls and powered by a wind-driven generator behind the propeller. The horizontal gyroscope kept the wings level, while the vertical one did the steering. Modern autopilot systems bear little resemblance to that rudimentary device. Controlled by onboard computers running immensely complex software, they gather information from electronic sensors and continuously adjust a plane’s attitude, speed, and bearings. Pilots today work inside what they call “glass cockpits.” The old analog dials and gauges are mostly gone. They’ve been replaced by banks of digital displays. Automation has become so sophisticated that on a typical passenger flight, a human pilot holds the controls for a grand total of just three minutes. What pilots spend a lot of time doing is monitoring screens and keying in data. They’ve become, it’s not much of an exaggeration to say, computer operators.

And that, many aviation and automation experts have concluded, is a problem. Overuse of automation erodes pilots’ expertise and dulls their reflexes, leading to what Jan Noyes, an ergonomics expert at Britain’s University of Bristol, terms “a de-skilling of the crew.” No one doubts that autopilot has contributed to improvements in flight safety over the years. It reduces pilot fatigue and provides advance warnings of problems, and it can keep a plane airborne should the crew become disabled. But the steady overall decline in plane crashes masks the recent arrival of “a spectacularly new type of accident,” says Raja Parasuraman, a psychology professor at George Mason University and a leading authority on automation. When an autopilot system fails, too many pilots, thrust abruptly into what has become a rare role, make mistakes. Rory Kay, a veteran United captain who has served as the top safety official of the Air Line Pilots Association, put the problem bluntly in a 2011 interview with the Associated Press: “We’re forgetting how to fly.” The Federal Aviation Administration has become so concerned that in January it issued a “safety alert” to airlines, urging them to get their pilots to do more manual flying. An overreliance on automation, the agency warned, could put planes and passengers at risk.

The experience of airlines should give us pause. It reveals that automation, for all its benefits, can take a toll on the performance and talents of those who rely on it. The implications go well beyond safety. Because automation alters how we act, how we learn, and what we know, it has an ethical dimension. The choices we make, or fail to make, about which tasks we hand off to machines shape our lives and the place we make for ourselves in the world. That has always been true, but in recent years, as the locus of labor-saving technology has shifted from machinery to software, automation has become ever more pervasive, even as its workings have become more hidden from us. Seeking convenience, speed, and efficiency, we rush to off-load work to computers without reflecting on what we might be sacrificing as a result. [Continue reading…]

Now if we think of automation as a form of forgetfulness, we will see that it extends much more deeply into civilization than just its modern manifestations through mechanization and digitization.

In the beginning was the Word and later came the Fall: the point at which language — the primary tool for shaping, expressing and sharing human intelligence — was cut adrift from the human mind and given autonomy in the form of writing.

Through the written word, thought can be immortalized and made universal. No other mechanism could have ever had such a dramatic effect on the exchange of ideas. Without writing, there would have been no such thing as humanity. But we also incurred a loss and because we have such little awareness of this loss, we might find it hard to imagine that preliterate people possessed forms of intelligence we now lack.

Plato described what writing would do — and by extension, what would happen to pilots.

In Phaedrus, he describes an exchange between the god Thamus, king and ruler of all Egypt, and the god Theuth, who has invented writing. Theuth, who is very proud of what he has created says: “This invention, O king, will make the Egyptians wiser and will improve their memories; for it is an elixir of memory and wisdom that I have discovered.” But Thamus points out that while one man has the ability to invent, the ability to judge an invention’s usefulness or harmfulness belongs to another.

If men learn this, it will implant forgetfulness in their souls; they will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks. What you have discovered is a recipe not for memory, but for reminder. And it is no true wisdom that you offer your disciples, but only its semblance, for by telling them of many things without teaching them you will make them seem to know much, while for the most part they know nothing, and as men filled, not with wisdom, but with the conceit of wisdom, they will be a burden to their fellows.

Bedazzled by our ingenuity and its creations, we are fast forgetting the value of this quality that can never be implanted in a machine (or a text): wisdom.

Even the word itself is beginning to sound arcane — as though it should be reserved for philosophers and storytellers and is no longer something we should all strive to possess.

Print Friendly, PDF & Email

9 thoughts on “Plato foresaw the danger of automation

  1. Christopher Hoare

    Another excellent article you have presented to us. (Of course your activity is a form of automation that makes us less able to find the articles for ourselves LOL.)
    I have seen the dumbing down of the human ability many times since I was an engineering student. The Brits use to ensure that the budding engineers could design and make the tools they needed themselves. Being able to file down a piece of steel and keep it perfectly flat was one of the earliest tests we had to pass. What of today’s designers—would they even know there is a skill there that must be learned?

    I recall there was a deliberate policy of requiring trainee pilots to learn on aircraft that were relatively difficult to fly during WWII as it weeded out those who were never going to be skillful enough for their future carreers. Which leads me to point out that the automatic transmissions in automobiles were a marketing ploy to allow incompetents to buy and use automobiles—and incidently create the appalling accident figures we have come to regard as inevitable.

    Oh lord—why would people want to submit to the intrusive hysteria in airports to trust yourselves to automated plastic aeroplanes? I’m glad I did all my necessary traveling when typical 100 seat airliners were never more than 75% full and were flown by pilots who took pride in their skills.

  2. John Merryman

    What was this wisdom Plato was afraid would be lost? Was it basic survival skills, elementary ingenuity and situational awareness, or something deeper and more connected? Knowledge is a function of making distinctions, but awareness is the connection we have with the world around us. Take as simple a concept as 1+1=2; If you actually add things together, you get another, larger thing, so we are adding the sets and getting a larger set, not the contents of the sets, or we have applesauce. Like the parts of our bodies and all its relationships, add up to a whole person, not just a sum of the parts. Yet we think the answers are in the parts, the distinctions, not the connections and we have an atomized society and view of nature. Knowledge is in the distinctions, while wisdom is in the connections.

  3. Paul Woodward

    “Knowledge is in the distinctions, while wisdom is in the connections.” That’s right — and that’s why most educational systems are screwed up.

    At the age of sixteen, I had to choose between arts and sciences — a frustrating choice since biology, geography, English, and art were my favorite subjects. It really doesn’t make sense.

    Take, for instance, the study of a flower. Why wouldn’t one want to engage it from every possible angle? Which is to say: to understand its form, life-cycle, habitat, and through acute observation be able to express its beauty through words, drawing or painting.

    Suppose the early naturalists who lay the foundations for modern biology and botany had been equipped with cameras instead of pencils and notebooks, would they have accomplished more? I suspect they would have accomplished nothing at all, since the speed with which they could capture images would have meant they never cultivated the keen powers of observation and the patience required for fine discernment.

  4. John Merryman

    I’m afraid I never even made it that far in school. Having grown up on a horse farm, school pretty much seemed, especially in the 70’s, to be trade school for desk work and I wasn’t going to be stuck behind a desk any more than necessary. So the more I became self educated, the more the civilized work seemed robotically confused. The three main observations I could make are;
    1; We look at time backward. The present doesn’t move from past to future. The changing configuration of what is, turns future into past. Physics compounds the confusion by reducing time to measures of duration and mixing them up with measures of distance. Ask yourself, does the earth travel some fourth dimension from yesterday to tomorrow, or does tomorrow become yesterday because the earth rotates?
    This makes time an effect of action/change, rather than the basis for it. As an analogy, time is to temperature, what frequency is to amplitude. If time really were a vector from past to future, you would think the faster clock would move into the future more rapidly, but the opposite it true. It age/burns/processes quicker and so fades into the past more rapidly. If people thouht that one through, they might slow down a little.
    As for Quantum theory using an external timeline, the cat is not both dead and alive, because it is the actual occurrence of events that determines its fate, as probability collapses into actuality. If any physicist wants to argue, we can take this over to FQXI( I entered this observation in last years contest, ‘Which of Our Basic Physical Assumptions Are Wrong?’ and came in about 50th of 350 entries, so a fair number of people did think it worth some consideration.
    2; Monotheism has it backwards as well. The absolute is basis, not apex, so a spiritual absolute would be the raw essence of being from which we rise, not an ideal form from which we fell. As a side note on that, the other day my daughter informed me from college that life is pointless and I observed to her that while we are far more motivated by desire, than vision, desire without vision won’t get you very far.
    3; The last point I think needs making is that money is a contract, but we treat it as a commodity, but if people understood it is a contract, the system would be far more stable and far less destructive. I wrote an essay on this that goes into some detail, as a post-script to the Occupy movement;–What-is-Your-Occupation?detail=hide
    Not to clutter your comments section, I’m just a regular follower of your blog and like your worldview.
    John Merryman

  5. Paul Woodward

    John – thanks for sharing that. I can’t comment on much of what you are saying, but I do want to note something you wrote in your Daily Kos piece:

    While we think of modern society and its technical advances as unprecedented, biology evolved equal, if not far greater levels of complexity many millions of years ago. Much of our social and many of our technical structures naturally mimic these biological processes, yet the biology is far more evolved.

    This point, that biological complexity vastly exceeds the complexity of human inventions, cannot be overstated.

  6. John Merryman

    Thanks Paul. It’s a bit like that. You try figuring out one problem and after peeling away all the layers and untangling all the relationships, find yourself in a different space entirely.
    I think I long ago resigned myself to humanity driving this bus over the cliff and mostly see it as an effort to put some ideas in the mix, for when whoever is putting things back together, doesn’t make as many of the same mistakes. That our best and brightest are most concerned with the very large, very small and very abstract, is reasonable, but there is a crisis of multiverses and super strings, than needs a reset. Junk out usually means junk in.
    While having the financial system run by a bunch of egomaniacal, amoral gamblers isn’t such a good idea and our spiritual models dominated by ideas that were cutting edge three thousand years ago needs to be revisited. Nature gives us a pretty long leash and we seem to be about to hit the end of it at a run. Won’t be much fun, but might go down in the character building category.
    Keep up the good work,

  7. John Merryman

    One thought to keep in mind is that unsuccessful resistance strengthens power, but refuting its premises is slowly corrosive.

  8. Paul Woodward

    Good point. There are many people in Syria, Palestine, Egypt, and elsewhere across the region and the world who should be reflecting on the outcome of unsuccessful resistance.

  9. John Merryman

    They don’t have any choice, other than to resist. It’s those of us with the space to reflect who should examine the logic and the consequences of that logic.
    In that essay, I made a point that religion is society’s vision of itself and its context, while government is its management of that situation. Now we know the two don’t always agree. Christianity was an underground movement for four hundred years, before being grafted onto an empire in decline, so there is a foundational distinction built into its history, of a division of church and state. Islam, on the other hand, was a very successful religious, social and political movement for its first seven hundred years, coasted on that success for the next six hundred and has only found itself left behind by the west, since the fall of the Ottoman empire, a hundred years ago. The consequence being they have not worked out some foundational cultural issues about the relations between religion and government. It not all due to pressures applied from the west, though that is also part of it.
    This is what I mean when you try to figure out problems, it gets to be turtles all the way down.

Comments are closed.