Five years ago at the blog: unfortunately, this isn't as relevant now that the hype storm over AI has passed

 Tuesday, June 26, 2018

I'm afraid even the Brothers Grimm would have found Bitcoin a little too fantastic

I'm edging closer to the notion that the tools which we would normally use to critique journalism are no longer up to the task of discussing the 21st century technology narrative. Instead, the appropriate methods are probably those of the folklorist. We are rapidly approaching the realm of the myth and the tall tale. Why not start thinking in those terms?

It is standard practice when discussing something like a Jack tale to list the Aarne–Thompson classification. For example, Jack in the beanstalk fall under the classification AT 328 ("The Treasures of the Giant"). We could do something similar with the vast majority of tech reported. TakeTheranos. This and other accounts of college dropouts supposedly coming up with some amazing innovation can be classified under "wayward youth finds magic object."

 I've been getting quite a bit of thought recently to how magical heuristics have come to dominate the conversation about technology and innovation, but the idea of actually treating the narrative as folklore didn't hit me until I read this:

The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003. It illustrates the existential risk that an artificial general intelligence may pose to human beings when programmed to pursue even seemingly-harmless goals, and the necessity of incorporating machine ethics into artificial intelligence design. The scenario describes an advanced artificial intelligence tasked with manufacturing paperclips. If such a machine were not programmed to value human life, then given enough power its optimized goal would be to turn all matter in the universe, including human beings, into either paperclips or machines which manufacture paperclips.[4]

    Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips. The future that the AI would be trying to gear towards would be one in which there were a lot of paper clips but no humans.
    — Nick Bostrom, "Ethical Issues in Advanced Artificial Intelligence", 2003

Bostrom has emphasised that he does not believe the paperclip maximiser scenario per se will actually occur; rather, his intention is to illustrate the dangers of creating superintelligent machines without knowing how to safely program them to eliminate existential risk to human beings. The paperclip maximizer example illustrates the broad problem of managing powerful systems that lack human values

Suddenly it struck me that this was just the magic salt mill ever so slightly veiled in cyber garb. In case you're not up on your folklore...

It is Aarne-Thompson type 565, the Magic Mill. Other tales of this type include The Water Mother and Sweet porridge.

Synopsis

A poor man begged from his brother on Christmas Eve. The brother promised him, depending on the variant, ham or bacon or a lamb if he would do something. The poor brother promised; the rich one handed over the food and told him to go to Hell (in Lang's version, the Dead Men's Hall; in the Greek, the Devil's dam). Since he promised, he set out. In the Norse variants, he meets an old man along the way. In some variants, the man begs from him, and he gives something; in all, the old man tells him that in Hell (or the hall), they will want to buy the food from him, but he must only sell it for the hand-mill behind the door, and come to him for directions to use it. It took a great deal of haggling, but the poor man succeeded, and the old man showed him how to use it. In the Greek, he merely brought the lamb and told the devils that he would take whatever they would give him, and they gave him the mill. He took it to his wife, and had it grind out everything they needed for Christmas, from lights to tablecloth to meat and ale. They ate well and on the third day, they had a great feast. His brother was astounded and when the poor man had drunk too much, or when the poor man's children innocently betrayed the secret, he showed his rich brother the hand-mill. His brother finally persuaded him to sell it. In the Norse version, the poor brother didn't teach him how to handle it. He set to grind out herrings and broth, but it soon flooded his house. His brother wouldn't take it back until he paid him as much as he paid to have it. In the Greek, the brother set out to Constantinople by ship. In the Norse, one day a skipper wanted to buy the hand-mill from him, and eventually persuaded him. In all versions, the new owner took it to sea and set it to grind out salt. It ground out salt until it sank the boat, and then went on grinding in the sea, turning the sea salty.

I realize Bostrom isn't proposing this as a likely scenario. That's not the point. What matters here is that he and other researchers and commentators tend to think about technology using the specific heuristics and motifs people have always used for thinking about magic, and it worries me when I start recognizing the Aarne–Thompson classifications for stories in the science section.