I probably should have worked in a Matrix reference

Following up on this week's sci-fi, folklore, magic theme.

From Internet of Bugs Carl Brown:

But predicting what humanity would do in the future is only the beginning of the powers of a Super-AI.

I’ll quote here from “The AI Does Not Hate You” by Tom Chivers about a future AI called “Basilisk”:

...the Basilisk is saying, ‘If you work to bring me about as fast as possible, I won’t create a perfect copy of your mind and torture it for billions of subjective years.’ (The argument is that since a perfect copy of your mind would essentially be you, this is equivalent to bringing you back to life.) In essence, a thing that doesn’t exist yet may be blackmailing you from the future, threatening to punish you for not working hard enough to make it exist.

I only wish I were kidding. This belief (which is often called “Roko’s Basilisk”16 ) is even more unhinged than it sounds, and it comes from the website of one of the authors of IF ANYONE BUILDS IT EVERYONE DIES, and that author “Eliezer Yudkowsky, banned discussion of Roko’s basilisk on the blog for several years as part of a general site policy against spreading potential information hazards.”17 Luckily for posterity, those discussions are retained on the Internet Archive18, and the original post can be found replicated elsewhere19, so if you really want to - although I don’t recommend it - you can follow the links I’ve included below so you can see what kind of crazy underlies all this SuperAI nonsense.

Now, I want to be clear - there are many different versions of Roko’s Basilisk, and there are some people in the Doomer community who will say they don’t believe in any version of it. Despite that, the story about the Basilisk is still indicative of the powers that rationalists ascribe to ASI [Artificial SuperIntelligence --  MP]. According to an article in Slate20: “Yudkowsky said that Roko had already given nightmares to several LessWrong users and had brought them to the point of breakdown.” The Basilisk is also often treated as at least plausible in public discussions of SuperAI. And when the people outside the AI community or industry are exposed to this idea, even if they are told not to take the idea that an AI would torture them seriously, they are still left with an impression of an impossibly powerful AI that conceivably could bring people back to life to torture them. This isn’t harmless - this sticks in people’s heads and colors their ideas about AI thereafter.

Though this may border on staring-into-the-abyss territory, let's take a moment and think about what actually terrorized all these True Believers.

We won't even try to examine the argument that a future AI would go to all this trouble to do something that couldn't possibly influence past events. Instead, let's focus on the threat itself. Imagine you were told that there was a possibility that someone will build a computer simulation where a character based on you would be tortured. Does this thought terrify you? This is literally what we're talking about here.

Admittedly, the possibility that characters in a computer simulation might be sentient and self-aware is potentially an unsettling thought (it's also an excuse to post one of my favorite Doctor Who moments below), but the fact that one of these characters has your name shouldn't be that big of a deal. But, of course, the people who are deeply disturbed by this, those who find it haunting their nightmares, are not approaching it from any kind of scientific or logical standpoint.

There are definite echoes of Harlan Ellison's I Have No Mouth, and I Must Scream. That's a story about a non-omniscient computer that has to settle for torturing the last surviving humans. There's a much older precedent to the basilisk.

This is a story of an all-powerful being who brings you back to life after you die and punishes you through all eternity for making it angry. The story of the Basilisk is one of a vengeful God casting you into hell, nothing more, nothing less.

As a non-religious person, I try to be respectful of faith based beliefs (and I will delete replies that strike me as disrespectful), but this is something different. this is people believing in magical things then trying to convince us and themselves that they are actually the true rationalists.

The sad part is that this belief in a cyber Hades is neither the silliest nor the stupidest aspect of the AI movement. The rot and the absurdity run throughout the whole damn thing. 

These people are as flaky as your grandmother's best biscuits.