Руководитель IT отдела, подразделения Inaccuracy on matters of fact costs […….

In an effort to increase the total value of joy in the universe, there could be an ethical drive to make sure something that can feel joy exists, and that it does so. But to prioritize human beings in this regard strikes me as slightly arbitrary. After some apocalypse that leaves almost no human beings alive, if we can choose between recreating the human race and setting it into a stable long lasting civilization, or to create a long lasting civilization of robots that feel 10 times as much joy as us, for what reason should we choose to create a human society? It strikes me as largely a preference for organic life, and organic life with the same base pair sequence as us. If we’re looking at the total amount of joy, a daring and adventurous scheme to put a virus into Skynet so that all terminators walked around in a state of bliss would, at least in the short term, would be far more effective than ensuring humankind’s survival. But if Skynet and the terminators can feel at best mild bemusement and changing them is infeasible, then viva la resistance!

In terms of future generations, there seem to be roughly two uses of the concept. One is that of the future generations we know will have to live with the world we leave them, a common environmental theme. I consider their well being important. The other is the more recent idea of future generations who might not even exist, and as I’ve stated I don’t care very much about whether they come to exist. And with regards to this second case and not the first, I also agree with the second belief you oppose, that morality is about the living.

The third however, that death is insignificant if nobody is left around afterwards, I have everything against. I’ve been collecting a number of thoughts and lines of arguments on why this is actually in conflict with so many of our values and ideas, but hadn’t thought of your suggestion to make an inspiring story out of the case, which I found very poignant and amusing in how starkly it displays what is actually being suggested.

Part of what I think could be done much better about attempts to raise awareness or concern for existential risks is to move away from this focus on the abstract concept of future generations. If that is what motivates a person then by all means, please be motivated! At the end of the day, the more creative and dedicated people working to prevent such events the happier I’ll be. In that regard if you feel my arguments against including future generations in moral considerations might harmfully affect the number of people working on these issues, provide an easy excuse for inaction and were unwise to voice, then please remove this comment and I’ll take no offense. If only I can do that, just ask me and I’ll take it down.

What I suggest is a focus on the horrific scale of an existential disaster, in terms of the loss of human beings living right now or at the time it would happen. While the defining characteristic of existential risks is their effect on us as a species, on what they say about whether things with our genes are walking around a thousand years later, I don’t think this is the most salient characteristic. What I consider most salient is 1) the unprecedented scale of the tragedy, 2) the universal or nearly universal nature of the tragedy, and 3) the intimate and personal nature of the tragedy.

By (1) I refer to how such an event would dwarf the events we consider most terrible in our history. 9/11 is a soft breeze, or the brush of a feather, in comparison to such things. You would need nearly a hundred WWIIs to equal the death toll of an existential disaster. The number of people on this planet is so large it’s a challenge just to gain a feeling for. If you were to watch a screen upon which passed a collage of 10 human faces each second, it would take you over 21 years to see the faces of the 6.7 billion people currently on Earth. Longer of course if you took breaks to sleep or eat. If it were 1 face a second, at conventional current life expectancy you’d be dead before getting halfway through.

Related to the scale in an actually somewhat opposite matter is what I perceive as this odd belief that tragedy becomes acceptable when it happens to enough people. Eliezer Yudkowsky’s essay in Global Catastrophic Risks on cognitive bias (http://www.singinst.org/upload/cognitive-biases.pdf) had some interesting things to say about this and was a large catalyst for my own explorations of the issue. One thought experiment I would suggest in regards to the belief that death only matters if we’re around to care, is to imagine (probably inaccurately) that Nazi soldiers experienced no compassion or remorse whatsoever for their activities in the death camps, and that further either the rest of the world never found out, or shared a belief that such things were entirely acceptable. I’m making the assumption that the moral issue comes from caring about the death and not just being aware of it, but in this hypothetical world do the millions of murdered Jews’ deaths truly become acceptable? They don’t care now that they’re dead, and neither would anyone else, so it would seem by that reasoning it was perfectly fine. It might be unfair to use the example of the holocaust, being so emotionally charged, but this is in fact the entire point. The holocaust would be trivial in comparison to the death of every man woman and child currently on this Earth, and I’m assuming even those who feel death is only negative if people care about it consider the holocaust a terrible thing. If they are true to their beliefs, they must further believe that it is only BECAUSE we find it terrible that it actually is, only because we can no longer interact with those lost people. The well being of the murdered Jews can’t enter into it, they’re dead.

I’ll agree that ONCE we are all dead we won’t care, and I think it’s this belief that can make the end of humanity seem tolerable, at least at a first glance. It’d be like we never existed. Critically however it will NOT be the same as having never existed. What bothers me is the transition, the death of nearly 7 billion people, the getting from point A to point B. Once a murder victim is dead they may no longer care one way or the other, but that doesn’t mean we don’t condemn the murderer, even if there’s no indication they’ll murder again and the victim had not a single relationship. Is your sole reason for avoiding your own death that it would cause distress to those who know you? If an utterly lonely man is alone in a forest and a tree falls and kills him, does that mean he didn’t mind the idea of death when he got up that morning? Now that he’s dead he doesn’t think anything of it, but that’s not the stage we’re at. If all human beings were dead THEN no human being would care, but we’re not there yet. We AREN’T all dead, and pretty much all of us feel pretty strongly about not being dead anytime soon.

By the universal nature of the disaster I refer of course to the fact that, presuming the disaster wasn’t tame enough that a differential in wealth was somehow enough to save you, it happens to everyone. In a true world pandemic, Zimbabwe does not suffer without Britain suffering. The US does not suffer without China suffering. Again if the disaster was tame enough and they got lucky, a group of people even as large as a country might just avoid it, but this seems unlikely and it’s not then properly an existential risk (presumably). This can create the free rider problem, but I think it ideally could also be a source of great unity. For many of these risks, especially AGI, it seems most likely that we either all make it through or we all don’t. To help ourselves is to help Belgium, and I like to imagine that a great deal of positive solidarity might arise in this sort of situation.

Much more applicable for motivational reasons is the last point, something of the opposite end of the universal nature of the dangers, that the end of the world is an extremely personal thing. When we say “all human beings might die” what this means is “you, everyone you’ve ever loved or cared for, and then everyone else too.” This is something else that, assuming people can handle it constructively, I feel should be focused on. To work against existential risks is to save your own life and the life of your family. I hope that this might militate against the free-rider problem by being important enough to people that “just” having someone else work on the problem is not enough. Discouragingly however I’m having difficulty coming up with a single example or metaphor. While in a rational analysis the death of everyone in addition to those you love should not be a discouraging factor towards action, I wonder if it might be best at times to deemphasize the global nature of the problem.

All in all I feel an appeal for the well-being of generations who would not otherwise exist is a fairly academic and abstract idea. That certainly works for some people, but from my experience many people realize they don’t care very much about that, and jump to conclude that they don’t care very much about the death of humanity. I think the fact that when we discuss disasters of such staggering scale the focus tends to shift towards genetics and humankind as a species is a large factor in why many people suddenly don’t seem to mind human death when it’s 7 billion people dying and not 7 thousand. Like all species, we aren’t built to deal with or care about the idea of our species’ continuation but rather the immediate and visceral concerns of ourselves and those we value. It is for this reason I believe the focus in existential concern should shift to the massive loss of the life that’s around us right now every day, or the loss of our children’s lives should we pass away before the most critical windows of danger. (Hoping and assuming the windows are only open wide for a limited period.)

My interpretation of many people’s reaction to existential risks can be summed up with the following ridiculous hypothetical situation. You’re sitting at home, maybe doing some extra work, maybe playing a game with your children if you have them (or maybe a lover if you don’t). Suddenly the door bell rings and a man greets you on the front step. “Excuse me, I’m an extremely polite psychopath and I was wondering if you would allow me to murder you and your family.” I think most would be liable to reply something along the lines of “like hell I will.” However a few weeks later again someone calls at the house, you open the door and are greeted with “Greetings on this fine evening! I’m part of a hundred thousand strong association of polite psychopaths, and we’ve got a mind to kill every human being before committing ritual suicide. Would you mind if we started with you and your family?” to which oddly the reply seems almost to be “Well, you know I never really cared a lot about human beings as a species or whether there are future generations after us, and I have to admit that once we’re dead we’re not going to mind, so sure come on in. You’ll find my family in the dining room.”

(I apologize if my any of my reasoning is fallacious, if it is please don’t hesitate to point that out. Pride is a pretty trivial thing next to the survival of the human race and thousands of years spent in a potential utopia.)