"Utopia forgives everything"

From Wikipedia

 TESCREAL is a neologism proposed by computer scientist Timnit Gebru and philosopher Émile P. Torres. An acronym, it stands for Transhumanism, Extropianism, Singularitarianism, (modern) Cosmism, Rationalism (denoting the ideology of the rationalist community), Effective Altruism, and Longtermism. Gebru and Torres argue that these ideologies should be treated as an "interconnected and overlapping" group with shared origins.^[1] They claim these constitute a movement that allows its proponents to use the threat of human extinction to justify expensive or detrimental projects and consider it pervasive in social and academic circles in Silicon Valley centered on artificial intelligence. As such, the acronym is sometimes used to criticize a perceived belief system associated with Big Tech. 

Long but fascinating interview of Emile Torres describing the literal death cult (no, seriously, I am not exaggerating) that strongly influences and arguably dominates the AI industry.

I've pulled some key passages. The excerpt is longer than I'd like, but about two thirds as long ass my first pass. All quotes from Torres unless otherwise indicated.

 And the debate right now, the public debate about the development of superintelligent machines, should be between Team Human and Team Posthuman, but Team Posthuman has more or less monopolized the conversation. They've hogged the microphone.

So you're exactly right. So, for example, I mentioned that Yudkowsky is not anti-anti or pro-human, even though if you read his book If Anyone Builds It, Everyone Dies, you kind of get that impression. But he has explicitly said in interviews that once posthumanity arrives, then it may be unethical to continue to have biological humans.

And he draws this, you know, quite offensive ableist analogy between a posthuman choosing to have biological humans or children—or humans continuing to have biological humans in the posthuman era—and someone right now intentionally choosing to have a disabled child, right?

And in the same interview, he talks about how, you know, it might be good just to wipe all humans off the Earth. In other interviews, he said that if the only and reliable way to create superintelligences flitting about the universe, quote, "having fun," is to sacrifice all of humanity, then he would make that trade-off, right?

So he's on Team Posthuman, not Team Human.

The debate has to do with AI safety, which is exactly that dispute, the tension between the accelerationists and the doomers. The question is how to go about developing superintelligence. Everybody agrees, including Yudkowsky, that we need superintelligence. Superintelligence, as Nick Bostrom says, virtually all paths to a utopian paradise in the future take us through superintelligence. So we can't not build it. The question is when to build it and how, right?

And so you have some people who are on the doomer side, in the doomer camp within the TESCREAL movement, who are screaming that we aren't yet ready to build superintelligence. We don't know how to align it with human values. But what they mean by human values is the TESCREAL worldview, right?

So, for example, if there were a superintelligence that were aligned with my anti-TESCREAL worldview, and if this superintelligence were to shape the entire future of life in the universe, which is what they imagine superintelligence ultimately doing, they would think that's an existential catastrophe, right? Because what they want is a cosmic posthuman utopia, right?

So it's not about aligning it to, like, human values. It's really about aligning it to TESCREAL, anything other than that. I mean, literally, the definition of existential risk is any event that prevents us from establishing this, you know, posthuman paradise among the stars.

...

So again, for lack of better terms, there is a difference between terminal extinction, as I call it, and final extinction.

Terminal extinction is just a scenario in which our species disappears entirely and forever, full stop. That's the only condition that needs to be satisfied for terminal extinction to have happened.

Final extinction kind of builds on this, and it denotes a scenario whereby our species disappears entirely and forever, and we do not leave behind any posthuman successors.

All the TESCREAL people care about is avoiding final extinction. That is it. Many of them hold a pro-extinctionist view, according to which we should bring about terminal extinction without bringing about final extinction. Which is just to say we should survive for long enough to create our posthuman successors. And then, once our posthuman successors are here, we should dip out. We should, you know, follow the dinosaurs and the dodo into the eternal grave of extinction, right?

And so it's just—I find it deeply frustrating to see the public conversation where people are like, you know, all of these TESCREAL guys, like Yudkowsky, who has deep roots in effective altruism and rationalism, which grew out of the libertarian transhumanist movement, talking about the importance of avoiding human extinction. Because most people sitting at home are interpreting that as, like, yeah, we should do whatever we can to ensure the continued survival of our species.

That is not what they're talking about. They're talking about ensuring we survive long enough to bring about the posthuman era. At which point, who cares what happens to the 8.3 billion people on Earth? You know, we're just irrelevant at that point. And from a grand cosmic perspective, we're utterly insignificant, right?

So our role in this whole eschatological scheme is ultimately to be the transitional species between the biological and digital realms, right? And that is our role. And once our eschatological duties have been discharged, there is no reason for keeping us around.

So this is a deeply anti-human position.

...

Just as a quick side note, I mean, long-termism tells billionaires exactly what they want to hear. It says not only are you morally excused from helping the poor or, you know, focusing on animal welfare, you know, here-and-now kind of problems, but you are a morally better person, a superior person, for focusing on the far future.

...

I mean, EA has been described by Molly Lieberman as a bait and switch, where it sort of draws people in by advertising that they care about, you know, alleviating global poverty. And then, once people are in, they say, "Well, actually, there's a much better way to do the most good: if you can positively affect a tiny, tiny percent of the trillions and trillions and trillions of future digital people spread throughout the universe, the amount of good you'll do, in terms of expected value, is just many orders of magnitude greater than if you were to lift the 1.2 billion people in multidimensional poverty today out of poverty."

...

NEWPORT

Interesting. So, I mean, is this going too far to say? Because this is what's occurred to me: when I think about this type of long-termist, expected-value-style utilitarianism, where, you know, an individual human's life—you move away from that sort of fundamental natural-law, religious principle at the core of, you know, all normative moral thinking, of humans being infinitely valuable, a human life being infinitely valuable.

When you move away from that liberal conception, immediately what comes to mind is the 20th-century utopian movements. So if you look at far-right fascism, if you look at the far-left movements, the mid-century movements were completely willing for many millions of people right now to die because, yeah, but the utopia they were going towards was worth it.

So if you're in Nazi Germany, you're like, well, yeah, maybe it's not the best. We have to kill, you know, six to eight million people, but it's worth it. What matters is it's going to give us this sort of utopian state, and that's worth more.

Or if you're in Soviet Russia at a certain point, like, sure, half of Ukraine is dying from starvation, but the point is you have to break a couple of eggs to make an omelet, and the omelet is going to be this sort of utopian, you know, socialist-communist world.

And so, as soon as you move away from the precept that life is—you know, a human is infinitely valuable and has their own natural rights—as soon as you move away from that, we've tried this experiment. It ends in a place that we just universally look back and say, "That was really bad."

Am I going too far to put this in the same ontological category within the world of ethics?

TORRES
In my opinion, absolutely not.

Newport closes with this:

The Nazis did not get the thousand-year Reich, but tens of millions of people died. The Soviet communists did not get their socialist utopia, but tens of millions of people died along the way.

So, we've learned this lesson: the pursuit of utopias where you're saying the value of humans alive right now pales in comparison to the value of the utopia that I am pursuing—we have seen that exact configuration throughout history. No one has ever successfully gotten to their utopia. Every time it's been pursued with any sort of well-founded vigor, it's been a blight on human history.