Podcast: Play in new window | Download | Embed
Subscribe: Apple Podcasts | RSS
ReWriting the Human Story: How Our Story Determines Our Future
an alternative thought experiment by Nikola Danaylov
Chapter 11: The AI Story
Computer Science is no more about computers than astronomy is about telescopes. Edsger Dijkstra
When looms weave by themselves, man’s slavery will end. Aristotle
Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended. Vernor Vinge, 1993
Today we are entirely dependent on machines. So much so, that, if we were to turn off machines invented since the Industrial Revolution, billions of people will die and civilization will collapse. Therefore, ours is already a civilization of machines and technology because they have become indispensable. The question is: What is the outcome of that process? Is it freedom and transcendence or slavery and extinction?
Our present situation is no surprise for it was in the relatively low-tech 19th century when Samuel Butler wrote Darwin among the Machines. There he combined his observations of the rapid technological progress of the Industrial Revolution and Darwin’s theory of evolution. That synthesis led Butler to conclude that intelligent machines are likely to be the next step in evolution:
…it appears to us that we are ourselves creating our own successors; we are daily adding to the beauty and delicacy of their physical organisation; we are daily giving them greater power and supplying by all sorts of ingenious contrivances that self-regulating, self-acting power which will be to them what intellect has been to the human race. In the course of ages we shall find ourselves the inferior race.
Samuel Butler developed further his ideas in Erewhon, which was published in 1872:
There is no security against the ultimate development of mechanical consciousness, in the fact of machines possessing little consciousness now. A mollusk has not much consciousness. Reflect upon the extraordinary advance which machines have made during the last few hundred years, and note how slowly the animal and vegetable kingdoms are advancing. The more highly organized machines are creatures not so much of yesterday, as of the last five minutes, so to speak, in comparison with past time.
Similarly to Samuel Butler, the source of Ted Kaczynski’s technophobia was his fear that:
… the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines decisions. As society and the problems that face it become more and more complex and machines become more and more intelligent, people will let machines make more of their decisions for them, simply because machine-made decisions will bring better result than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won’t be able to just turn the machines off, because they will be so dependent on them that turning them off would amount to suicide. The Unibomber Manifesto
As noted at the beginning of this chapter, humanity has already reached the machine dependence that Kaczynski was worried about. Contemporary experts may disagree on when artificial intelligence will equal human intelligence but most believe that in time it likely will. And there is no reason why AI will stop there. What happens next depends on both the human story and the AI story.
For example, if AI is created in a corporate lab it will likely be commercialized. If AI is created in a military lab it will likely be militarized. If AI is created in an Open Source community it will likely be cooperative and collaborative. And if it is created in someone’s garage it will likely reflect the story of that particular person or people. So, the context within which AI is created will shape its own origin story and that story will define the way it treats humanity.
Thus a military AI will likely treat humans as allies and enemies. A commercial AI will likely treat humans as customers and products. An Open Source AI might treat humans as parents, coders, friends, or allies. [Given current funding trends the first two types seem the most likely.] So the most crucial thing that humanity will do when creating AI is narrating the AI origin story. Because similar to us, by describing how it came into being, what it is here for, what’s its purpose and proper place in the universe, its story will determine its future. If we get the AI story right we have a chance to coexist peacefully. But if we get it wrong that could mean a Gigawar of planetary proportions, even extinction.
For example, what if, like us, AI ends up with a story of being the pinnacle of evolution, the smartest species in the universe, and, ultimately, God?
This “AI-ism” is going to be to the AIs very much like what humanism is to us, humans. Would it be surprising if it uses this AI story to justify enslaving and killing billions of humans and destroying the biosphere?! Just like we are using ours to justify killing 73 billion animals and 1.3 trillion aquatic organisms every year. Because as Zora Neale Hurston pointed out in Tell My Horse:
Gods always behave like the people who make them.
That is why it is crucial that humanity transcends humanism to embrace post-humanism, post-anthropomorphism, post-exclusivism, and post-dualism. Because one specieist story should not be replaced by another. Not even with one about the primacy of the superintelligent machines, transhumans, posthumans, aliens, or uplifted animals. To be a true revolution, the AI revolution must change our story structure and abolish its current hierarchy. Otherwise, oppression will persist, suffering will increase and the only change will be who is the oppressor and who is oppressed.
In other words, the dangers posed by AI originate in the same place as the dangers posed by humanity: our story. If like the human story, the AI story ends up as one of uniqueness, exclusivity, progress, supremacy, dominance, alienation, teleology, manifest destiny, and godhood then we can expect a similarly destructive impact. Therefore, to minimize suffering and improve our own chances of survival during the turbulent 21st century, both humanity and AI must embrace a new type of story. One that is decentralized, non-singular, non-hierarchical, non-specieist, non-dualistic, and non-exclusive. Because a multiplicity is ethically better than a singularity. And because it is safer too.
Julie Shah is a roboticist at MIT and an associate dean of social and ethical responsibilities of computing. She directs the Interactive Robotics Group in the Schwarzman College of Computing at MIT. She was a Radcliffe fellow, has received an National Science Foundation Career Award, and has been named one of MIT Technology Review’s “Innovators Under 35.” She lives in Cambridge, Massachusetts.
Laura Major is CTO of Motional (previously Hyundai-Aptiv Autonomous Driving Joint Venture), where she leads the development of autonomous vehicles. Previously, she led the development of autonomous aerial vehicles at CyPhy Works and a division at Draper Laboratory. Major has been recognized as a national Society of Women Engineers Emerging Leader. She lives in Cambridge, Massachusetts.
Juan Enriquez is a leading authority on the economic impact of life sciences and brain research on business and society as well as a respected business leader and entrepreneur. He was the founding Director of the Harvard Business School’s Life Sciences Project and is a research affiliate at MIT’s synthetic neurobiology lab. After HBS, Juan became an active angel investor, founding Biotechonomy Ventures. He then co-founded Excel Venture Management. Author and co-author of multiple bestsellers including As the Future Catches You: How Genomics Will Change Your Life, Work, Health, and Wealth (1999), The Untied States of America: Polarization, Fracturing and Our Future (2005), Evolving Ourselves: Redesigning Humanity One Gene at a Time (2015,) and 
I have previously interviewed a few fantastic scientists and philosophers but rare are those strange birds who manage to combine together both deep academic training and the living ethos of those separate disciplines. Prof. 
People often ask me about my most favorite interview I have ever done. And my usual reply is that interviews are like children, even if we have our favorites it is not wise to express that outwardly because all kinds of problems will follow. And yet, after having published nearly 250 episodes of my podcast, I can hardly remember one that has had a greater impact on me than my 
Karl thinks that AI is a bit of a red herring; complex systems and predictability; why Global Warming is not a problem to be solved but a constraint to work within; pre-apocalyptic moments as a possibility to create something new; why code is law and technology is a value; 
It’s been 7 years since my 
Gary Marcus is a scientist, best-selling author, and entrepreneur. He is Founder and CEO of 
A few weeks ago I got interviewed by Ricardo Lopes for 