Top 10 Reasons We Should Fear The Singularity

Why do we fear the technological singularity?

Well, let me give you what I believe are the top 10 most popular reasons:

1. Extinction

Extinction is by far the most feared as well as the most commonly predicted consequence of the singularity.

The global apocalypse for the human race comes in many flavors but some of the most popular ones are: the supersmart terminator AI’s – a robopocalypsenanotechnology gone rogue – the so called grey goo scenario, home-made Smart Weapons of Mass Destruction – used by terrorists and nihilists; genetic modifications or mutations – turning us into living-dead zombies; science experiments gone wrong – the Large Hadron Collider creating a black hole that engulfs the planet…

In short, the fear is that, as Bill Joy notoriously put it: The Future Doesn’t Need Us.

2. Slavery

Perhaps the second most common reason for fearing the singularity is the potential slavery or subjugation of the entire human race. The argument is pretty straight forward:

Once we have super smart AIs we stop being the smartest entities on this planet. In other words, we have created Gods while remaining mere humans. So, if for whatever reason the machines decide not to exterminate us, then, chances are that, since they will be vastly superior to us, they will enslave us. This can be accomplished in a variety of ways: either explicitly – with us being aware of our bondage, or implicitly – without us realizing it (the Matrix/simulation scenarios).

3. World War III – Giga War

The third most common fear of the singularity is, of course, World War 3. A Giga War of unprecedented scale, sophistication and efficiency of death and destruction that may be the result of either the clash between the human race and the AIs or between different fractions of humans: e.g. the ArtIlect War of terrans versus cosmists as foreseen by Hugo de Garis. Whatever the case may be, it will likely result in billions of deaths and a collapse or complete eradication of our civilization.

4. Economic Collapse

Some have argued that, if we somehow manage to avoid the previous three scenarios, then, we are likely to experience a complete economic collapse:

The complete robotization of our society is likely to lead to overproduction of goods and services. Yet, since it is claimed that most people will lose their jobs to the robots, there will be a global unemployment rate of unprecedented scale which in turn will collapse the demand for those robo-made goods and services. Combine this with a population explosion of 9 or even 10 billion mostly unemployed people who have no means to create income and buy anything, and we are looking at a global economic collapse.

5. Big Brother AI

This scenario is a milder version of the slavery/subjugation Matrix option because we are still under the complete control of an all-knowing Artificial Intelligence. The main difference here is that the AI is merely doing what is best for us, rather than what is best for it: we have a benevolent, omnipotent, absolute monarch protecting us from our worst enemies – our fellow human beings and our own selves. It is all done in the name of maximizing security, prosperity and overall happiness for all people across the planet. The only minor negative is a little bit of propaganda and ideological, political or religious brainwashing required to prop up “the cult of the AI,” but that’s OK since it is for our own good.

6. Alienation and Loss of Humanity

Following the “if you can’t beat them, join them” maxim, one way of potentially surviving the singularity is by merging with the machines. This idea – that we can and should improve on what we have been handed down by mother nature, is often referred to as transhumanism. Merging man and machine via biotechnology, molecular nanotechnologies and artificial intelligence, we would increase our cognitive abilities, physical strength, emotional stability and overall health and longevity.

The fear, of course, is that by doing so we are going to lose the very essence of being human – our human nature, our human souls and human identity. Furthermore, at the collective level, the loss of humanity will also mean alienation or loss of community which is to say that the resulting variety of posthuman entities will be so wide apart as to negate any connection whatsoever among different individuals. This in turn will mean that humanity, in fact, did not survive but succumbed to the machine invasion and indeed went extinct.

7. Environmental Catastrophe

Our history shows that our environmental destructiveness is in direct proportion of our technological prowess. Once we live in a global society where everything is mass produced by robots, our manufactured civilization will sever the last connection to the natural world. We will lose the very last bit of respect for mother nature:

Why preserve the rain forest if we can create a “better” and “smarter” one? Why care about biodiversity, species’ extinction or environmental degradation if we can revive and mold those for our own purposes or pleasure?

Why care about anything if we are (technological) Gods?!

8. Loss of History, Knowledge and Spatial Resolution 

The ever accelerating process of digitization comes along with a certain loss or even destruction of data. This data can be in the form of history, cultural traditions, dead languages or important scientific information. For example, NASA recently admitted it has lost the ability to recover much of the computer data from some of the Apollo missions and the Moon landings. Thus certain kinds of vitally important and unique knowledge as well as history or cultural traditions are lost forever. To know if we are getting a good deal or not, we must fist quantify the data losses and compare them to the potential gains. Yet, at the break-neck speed we’re moving forward few have time for such calculations.

It seems that we live in an analog universe with infinite resolution – both zooming in and out, as far as we can. The process of digitization captures a mere fraction of it. Just like a compressed .mp3 file captures only a part of the actual musical performance, this process creates symbols which are digital representations of the real thing. The fear is we may end up losing awareness that the digital realm is a realm of symbols – a mere reflection of the true analog universe, ending up in Plato’s Digital Cave of Illusions.

9. Computronium and Matrioshka Brains 

As far as we can tell it seems we live in a universe full of dumb matter. This, of course, makes for a pretty dumb universe too.

However, extrapolating from our own development, it would appear that as time goes by there is a movement from less towards more intelligence in the universe. Thus, given enough time, more and more of our planet and, eventually our universe, is likely to contain and consist of more and more intelligent matter. This process is likely to continue until Moore’s Law collapses and an equilibrium is reached. Such a theoretical arrangement of matter – the best possible configuration of any given amount to achieve a perfectly optimal computing device, is the substrate also known as computronium.

A Matrioshka brain is a hypothetical megastructure of immense computational capacity. Based on the Dyson sphere, the concept derives its name from the Russian Matrioshka doll and is an example of a planet-size solar-powered computer, capturing the entire energy output of a star. To form the Matrioshka brain all planets of the solar system are dismantled and a vast computational device inhabited by uploaded or virtual minds, inconceivably more advanced and complex than us, is created.

So the idea is that eventually, one way or another, all matter in the universe will be smart. All dust will be smart dust, and all resources will be utilized to their optimum computing potential. There will be nothing else left but Matrioshka Brains and/or computronium…

“NASA are idiots. They want to send canned meat to Mars!” Manfred swallows a mouthful of beer, aggressively plonks his glass on the table. “Mars is just dumb mass at the bottom of a gravity well; there isn’t even a biosphere there. They should be working on uploading and solving the nanoassembly conformational problem instead. Then we could turn all the available dumb matter into computronium and use it for processing our thoughts. Long-term, it’s the only way to go. The solar system is a dead loss right now – dumb all over! Just measure the MIPS per milligram. If it isn’t thinking, it isn’t working.” (Accelerando by Charles Stross)

10. Fear of Change

Fear of change and fear of the unkown are deeply embedded in the human psyche: We all want to be comfortable. Not knowing is very, very uncomfortable. Realizing that the coming change is radically unique – both in scale and unpredictibility, is even more discomforting.

When it comes to survival nobody likes surprises. So we take it as a matter of both personal as well as collective security to model and at least roughly foresee the future.

The singularity is a radical change of arguably cosmic proportions which is by definition impossible to model, let alone predict. Thus, there is no surprise it evokes very deep insecurity and primal fear.

The question is: Are you afraid?! Are you not very, very afraid?!

  • Diego

    I am very very concerned. We live in a binary century, a 0 means we are so doomed it is hard to think about. A 1 means we are so lucky the Iron Man can’t imagine our awesomeness.
    The strange thing though is to recognize that we can’t stall history, and can’t stop both from happening. One of them will, and we can only steer, we cannot frenate.

  • http://www.singularityweblog.com/ Socrates

    Very well put Diego, very well!

  • http://www.facebook.com/people/H-Keith-Henson/712965641 H. Keith Henson
  • http://twitter.com/33rdsquare 33rdsquare

    Great list Nikola. A lot of food for thought. Should we really fear a Matrioshka Brain? A while back I posted some others:
    http://www.33rdsquare.com/2012/01/top-five-singularity-concerns.html

  • MatthewJPrice

    Thanks for posting Nikola. Since I have time, I’m just going to go down the list with you :)

    1: A risk we cannot ignore, certainly. I think this is probably the only one on your list I fully agree with.
    2: I find it unlikely we would be needed for very long. This turns into 1.
    3: This would mostly likely be pre-singularity, or it wouldn’t be much of a fight. Not worth calling it a war if one side just goes “pop” and falls over. depending on your definition of human, this turns into 1.
    4: This is a definite pre-singularity. In fact, probably the mid 20s. Robotics and AI will wreak havoc with employment which destroys the whole system. Lots of literature on this but my favorite is “Lights in the Tunnel”.
    5: I’m strangely comfortable with this, especially if we’re allowed to transcend to a state that does not need babysitting. If our best interests are at its heart, this is certainly the case.
    6: This one strikes me as irrational. Did our hominid ancestors fear losing their nature? We’ll do what’s best for our selves, however we see that. I have full confidence I’ll be much happier as whatever it is I’m going to be than whatever it is I am now.
    7: There’s a great deal of reason to worry about this now, but after the singularity, I don’t think it will be an issue. “any sufficiently advanced technology is indistinguishable from nature” is a quote that doesn’t quite capture it but puts across the general feel. Yes the earth will be different, but it will actually be greener and healthier than it is now.
    8: I am concerned about this one, but I expect enough people will care that any historians with suffiiciently advanced tools will be able to do a great deal of repairs to the loss.
    9: I don’t see the downside of this…
    10: also a non starter for me

    So… I hope we don’t go extinct! Thanks again for sharing

  • http://profile.yahoo.com/Y4THMOJSECS75CYPT5VYE6XZBU Bill

    God will return before mass extinction reaches this level, but it will get bad before his return. and when the Lord returns who will he find working for his gospel, not very many. there will be weeping and gnashing of teeth, please people repent now and god will bring us through the storm to meet him. the bible states there will be his children in the last day who will see him come on the clouds of glory , so there you have we will not all pass away like so many will have you believe

  • cosimoto

    Are the smartest people in the world stupid? I don’t think so! We are already becoming immersed in the Matrix as mentioned above. Only the more immersed we are the less apparent it becomes. Also, this article uses terms such as slavery and big brother. These concepts are as crude as stone knives and bear skins. These people are smarter than that. They won’t keep slaves! They will use “human resources”… a term already well accepted in society. And big brother… he is not needed when the machines YOU control for YOUR pleasure slowly, gently guide you to behave the way the man behind the curtain wants you to. Finally, do you think these people will create machines that they can’t turn off? If they convince everyone that these computers can’t be controlled then people will wind up doing something analogous to being date raped and blaming the toaster. Don’t fear powerful machines. Fear powerful men!

  • BillybonesIII

    Too many wise guys spouting technical terms spawned in the last 20 odd years. Pathetic to think that all this wisdom goes down the terlet. Scrabbling and scraping to live a few more years. For what? oh, wait! Maybe there’s an app for eternity. Pull your heads out of the TV matrix and take a look in the Book that does have all the answers. Singularity…….give me a break!

  • Bayaba

    Being married to a part Quechuan (one of the sects of the Mayan people) woman, I understand a bit of what the Mayan people believe about the end of the current cycle of their calendar. They believe this coming December will mark the end of duality and the beginning of singularity. To them, singularity would be the understanding that all of humanity, in fact everything that exists, is one, is all connected, is singular. Everything any part of the whole does has an effect on the rest of the whole. Humanity has failed to see that on a broad scale to this point in time. As a result, many people serve their personal interests (the owners of the central banks of the world come to mind and no, Bill Gates is not one of the richest, financially speaking, people on earth, not by a long shot) without any consideration for how it will impact others. It’s time for that to end and I am very much looking forward to it.

  • http://vitamind3info.blogspot.com/ Adrian

    The bad stuff is only corruption and greed, Utopia is possible , we just neede to get them to adhere to the law, we’ve come so far, keep exposing the wrongdoing.

  • jollymonsam

    I see the light at the end of the tunnel OH shit, its the headlight of the train wreak that gonna happen in this NOV

  • advancedatheist

    Sheesh, people, give it up. You’ve fallen behind the times and talk like you still live in the 1990’s. The smart people like Tyler Cowen, Peter Thiel and Neal Stephenson have wised up to the fact that we live in an era of technological stagnation, not “acceleration.”

    Also read:

    http://www.wfs.org/blogs/dale-carrico/unbearable-stasis-accelerating-change

  • http://twitter.com/MarkRiva Mark Riva

    I am beyond afraid. I’ve gone into my fear and found great opportunity for growth. I realized I can’t get rid of fear until I do that. I’ve felt the fear and used its energy to move through it. I am no longer living in fear even though I know there are many reasons to be fearful. I just choose to be fearless instead.

  • http://www.singularityweblog.com/ Socrates

    Hey Mark, I believe you have found the secret to a life that is worthwhile and will make a difference!

  • Pingback: Hugo de Garis on Singularity 1 on 1: Are We Building Gods or Terminators?

  • Pingback: George Dyson on Singularity 1 on 1: In Wildness Is The Preservation Of The World

  • take it to another forum

    Enough with the irrationality.

  • Pingback: Top 10 Reasons We Should NOT Fear The Singularity

  • Tracy_R_Atkins

    10 reasons against, but there are 10,000 for it.

  • http://www.singularityweblog.com/ Socrates
  • Tracy_R_Atkins

    I had completely missed the 10 reasons for article. Thank
    you very much for pointing me to it. After looking it over, I can’t help but feel a bit excited!

    Thanks for the link!
    Good stuff indeed!

  • http://www.singularityweblog.com/ Socrates

    Very happy you like it friend!

  • http://pulse.yahoo.com/_HGJIR72S2FX2OMCYUQEV4PMWUU WilliamS

    Sounds absolutely moronic.

  • http://www.singularityweblog.com/ Socrates

    How so William?!

    Some of the smartest singularitarians say that those are legitimate concerns to have. And, how about the opposing article:
    http://www.singularityweblog.com/top-10-reasons-we-should-not-fear-the-singularity/
    Is this any less “moronic”?

  • http://twitter.com/haig haig

    1.) Extinction: This is an inevitability without the singularity, so given a certain extinction and a chance to escape that scenario, I’ll take that chance with the singularity. There is no going back.

    2.) Slavery: This is due to a misconception of the goal systems of AIs or technocratic control systems. I think the fear is overstated, that future global management systems will be more humane and egalitarian by necessity, and eventual AGIs will be much more moral than we are. Slavery is what evolved animals engage in, not designed humane systems.

    3.) World War III: This is a possibility we still have right now under our current political system without resorting to AIs or anti-transhumanist vs transhumanist fantasy scenarios. AIs vs humans is a hollywood trope that shouldn’t be taken seriously, and Hugo De Garis’s scenario is fear mongering without detailed understanding.

    4.) Economic Collapse: I’d state it as economic evolution. With things becoming more abundant and more automated, yes, compulsory ‘jobs’ will be mostly obsolete, but that is only a problem in our current system. Why do all these future scenarios depict radical disruption of all sectors and industries, yet maintain our current economic system as frozen? Lack of imagination or dogmatism I’d guess.

    5.) Big Brother: We have this already to a certain extent, and yes it has the possibility of getting much worse, but the trend of openness and transparency in information technologies is a strong countervailing force and, because future economic advancement relies on open systems, it seems were are locked in a dance that has openness winning out over closed, proprietary systems which big brother systems need to function.

    6.) Alienation/Loss of Humanity: What is a human? What if our nearest non homo sapien common ancestor decided not to evolve any further? We are part of an evolving universe, and what comes next is not a loss but a gain, a larger possibility space of experiences. Within that possibility space, those who choose to experience life as we know it currently will have that option, but I suspect that will be just a small fraction of the experiences available.

    7.) Environmental Catastrophe; Again, we’re headed for this right now, more or less. The biosphere is resilient, more resilient than humanity. It has survived extinction level events in the past and recovered, but humans, being the leading branch of the biosphere shooting out from the evolutionary bush, is on trajectory to transcend most of its evolved material condition and once that occurs, will be able to care for the biosphere much better than we can currently given our reliance on inefficient and unsustainable processes for our survival.

    8.) Loss of history/knowledge/resolution: This one made me chuckle. Long-term data reliability is a concern, but it is an important enough economic/business problem that we have and are advancing storage techniques that are robust and reliable. Knowledge has never before been so accessible, so detailed, so scrutinized, I can’t understand how one can say this trend will not continue. As for digitization not being detailed enough to capture the resolution of our analog universe, well, that is funny as well since with advances in digital computing comes advances in our ability to sample to ever more precision. Also, analog and quantum computing that don’t necessarily use discrete states (neuromorphic architectures etc.) are advancing right along with digital technologies.

    9.) Computronium: This is a speculative extrapolation of Moore’s law and is not necessarily the only outcome of a singularity. But, in the spirit of such a scenario, I’ll just say that as the universe can be seen as doing computations outside of our control already, why not rearrange it for computations more conducive to life and intelligence, whether that is in the form of computronium, black holes, or something else? Living in a matrix that we opt-in for is a much more desirable existence than living in this world we are thrusted into subject to the whims of nature and luck. The matrix is only a problem if you are there by force or deception.

    10.) Fear of change: Humans adapt, we do it better than any other organism. Yes, change is scary, but the alternative, stagnation and the closing of the future, is a much more scary and bleak dystopian scenario. Tell an illiterate villager 300 years ago that one day they can cure his trembling or his heart problems by implanting a gadget into his body, controllable over telephone lines, and he’d be terrified. Today, it is routine procedure that no one would turn down out of fear. The shock level will be larger than previous changes, sure, but people will realize how much better life is and accept the changes.

  • http://www.singularityweblog.com/ Socrates

    Great points friend. Really appreciate you taking the time to write such detailed counterpoints. I have to admit that I honestly agree with pretty much most of what you say. I wrote this article because these are some of the most popular fears of the singularity. Still my position is better reflected in the follow up article titled Top 10 Reasons We Should NOT Fear The Singularity – http://www.singularityweblog.com/top-10-reasons-we-should-not-fear-the-singularity/

    Feel free to let me know what you think about that one too ;-)

  • http://www.singularityweblog.com/ Socrates

    Well, I am familiar with both of these concepts John. And the point is that both of them will be negated given that we have sufficiently advanced technology. For example, peak oil only matters in oil based economies. If we have a serious break-trough in alternative energy e.g. solar, then, peak oil will not make one bit of a difference for us…

  • guy369

    All the points in this article are already happening today

  • myc o’ nutrients

    as a friend of, no not mine, ours, said “don’t you fumble, just be humble” All the best of futures for the future. err thats it

  • Pingback: The Great H+ Hoax: Nothing Positive About The New H+ Digital Series

  • Pingback: The Very Best of Singularity Weblog in 2012

  • Raiden

    The biggest reason I fear the Singularity? It would disprove the existence of the Human soul. If we can upload our minds into a machine to give us immortality, what does that mean for me? If the soul and mind is just information, no different and not in any way ‘special’ from an MP3, What does that mean for human being special in the scope of the cosmos? Modern science all but says we aren’t special, we evolved from apes etc. the singularity would just be the final blow to proving that opinion.

    I still prefer an Illusion of human exceptionalism, to the cold hard truth of the Singularity.

  • http://www.singularityweblog.com/ Socrates