Have we mentioned how unfair it is to single out Arthur Conan Doyle?

Yes, I'll admit that falling for those fairy pictures was pretty embarrassing, but Doyle had lots of company in his belief in the paranormal, including some remarkably distinguished late 19th and early 20th century names.

Brigit Katz writing for Mental Floss:

Marie and Pierre Curie readily admitted that nature was rife with mysteries that scientists had yet to identify and study. “[W]e know little about the medium that surrounds us, since our knowledge is limited to phenomena which can affect our senses, directly or indirectly,” they wrote in 1902, acknowledging that they did not fully understand the origin of radioactive energy.

Pierre was particularly fascinated by the paranormal. Introduced to spiritualism by his brother, the scientist Jacques Curie, he confessed in an 1894 letter to Marie that “those spiritual phenomena intensely interest me.” He believed that the paranormal realm was relevant to “questions that deal with physics,” and according to biographer Anna Hurwic, thought that spiritualism might uncover “the source of an unknown energy that would reveal the secret of radioactivity.”

... 

Just a few days before his death in 1906, Pierre wrote again to Gouy describing the last Palladino séances he would ever witness. “[T]hese phenomena really exist and it is no longer possible for me to doubt it,” he proclaimed. “There is here in my opinion, a whole domain of entirely new facts and physical states in space of which we have no conception.”

Marie does not appear to have been as intrigued by Palladino as her husband, according to Susan Quinn, author of Marie Curie: A Life. She had other demands on her time and energy, including her two young children and the intense public attention that followed her Nobel Prize win. But at the very least, Marie doesn't seem to have come away from Palladino’s séances as a firm disbeliever in the possibility of a spirit world—because after Pierre died, she continued to communicate with him.

 

Curie had company. William Crookes and fellow Nobel laureate Charles Richet were two of the many scientists who devoted a large part of the time to paranormal research. Albert Einstein wrote the preface to Mental Radio. Respected publications published serious discussions about psychic and supernatural phenomena. Not everyone was a believer, but it was rare to find people in the "absolutely, positively no-way" camp either.

Nor were the Curies the only ones to make a connection between the fantastic scientific discoveries of the day and the possibility of the paranormal. When Edison stumbled on the first radio transmitter back in the 1870s (that’s another story we need to get into), he wondered if it might explain telepathy. When the later debunked N-rays were announced, scientists speculated that they too might explain psychic phenomena.

When you are constantly bombarded with news of the seemingly magical—be it cylinders that capture voices, strange rays that let doctors see through flesh, or communications that somehow travel instantly through the air—it would probably seem foolhardy to prematurely dismiss the possibility of something outside of our range of knowledge.

We, on the other hand, have two large bodies of knowledge to draw on when evaluating claims of the supernatural. First, we have well over a century of paranormal research where, when you take out the frauds and the fatally flawed test designs, you find no support whatsoever for these claims. Second, we have a century of research into physics that has effectively ruled out any possibility of these forces causing ghosts to walk or psychics to tell you anything that couldn't be learned from a cold reading.

You and I aren't smarter than Einstein  or the Curies, but we do know a bit more.

Clive James's example may be god-awful, but what about the point he was trying to make?

 Picking up on our discussion of this claim:

The best Hitchcock film was directed by someone else. Charade would not be as good as it is if Hitchcock had not developed the genre it epitomises, but Hitchcock could never have created a film so meticulous, plausible, sensitive, light-footed and funny.

Everyone now seems to agree that "the best Hitchcock film" is, at best, quite a stretch, but what about the broader claim that a [blank] film/book/song might be the work of someone other than [blank]?

There are lots of examples where imitations were better than the original and where plagiarists, from Robin Williams to Shakespeare, put out work superior to what they were imitating, but that's not exactly what we're talking about here. 

In this context, the term "Hitchcock film" effectively defines a subgenre (think North by Northwest—with the important caveat that not every film Hitchcock made qualifies as a Hitchcock film by that standard.

Saying someone defines a subgenre is a bit of a left-handed compliment. Obviously, you have to be successful and influential to get there, but that success and influence largely almost always exist within a larger genre. It also suggests that someone else could do it. While Charade is a silly example, it’s not that difficult to imagine someone else theoretically making a better Hitchcock film than Hitchcock. I don’t think you could talk about a Kubrick film in the same way. That said, it is worth noting that countless enormously talented filmmakers—in some cases, arguably more talented than Hitchcock himself—have tried their hands at the subgenre and, as far as I can tell, have all fallen short. François Truffaut, Roman Polanski, and Brian De Palma all come to mind.

What about in other media? An Agatha Christie mystery would certainly qualify as one of these personal-brand subgenres, and we could probably find someone to argue that Ngaio Marsh wrote better Christie books than Christie did (I’m not taking a position on this one, I'm just saying someone might) but it's a difficult point to argue. I would be more than willing to make the case that Dorothy L. Sayers wrote better novels, but here we get into one of the big problems with "better [blank] than [blank]" claims: if you improve too much on the original, at some point, it ceases to be a [blank] work. (Tellingly, if probably unintentionally, when Kenneth Branagh wanted to make Hercule Poirot more modern and three-dimensional, he did so by giving him the backstory of Lord Peter Wimsey.) Sleuth also comes to mind. It plays with the conventions of an Agatha Christie story but mainly to subvert them.

If you're good enough to have a subgenre named after you, usually you are good enough to outshine your imitators, but I finally came up with an exception—one so obvious I don't know why it took me so long to think of it. A writer whose very name is a widely used adjective, arguably one of the most influential writers of the 20th century, and yet someone who was routinely outdone at his own game.

H.P. Lovecraft wasn’t a very good writer. There were good, even sometimes great, elements in his stories, but the stories themselves never rose above mildly inept. I went back and reread some Lovecraft, starting with Dagon, and with the exception of a few passages, it took me back to my days teaching junior high English.

We won’t even get into the racism and anti-Semitism.

Lovecraft's writing often comes across as a crude first draft of what could be a very good piece of fiction in the proper hands, which may be why we saw an extraordinary group of talented writers picking up his ideas and running with them—even as he was still writing.

Although the Mythos was not formalized or acknowledged between them, Lovecraft did correspond, meet in person, and share story elements with other contemporary writers including Clark Ashton Smith, Robert E. Howard, Robert Bloch, Frank Belknap Long, Henry Kuttner, Henry S. Whitehead, and Fritz Leiber—a group referred to as the "Lovecraft Circle".^[16][17][18]

Everyone named in that paragraph was a much better writer than H.P. Lovecraft, and it is because of them—and the others who followed—that his works are better remembered today than The Great God Pan or the stories of Lord Dunsany.

 

A Blacker Black Box

From Matt Levine's newsletter:

There are two basic ways to use artificial intelligence to predict stock prices:

1. You build a deep learning model to predict stock prices: You set up a deep neural net, you feed it tons of historical data about stocks, and you train it to figure out how that data predicts stock price returns. Then you run the model on current data, it predicts future returns, and you buy the stocks that it thinks will go up.
2. You take some deep learning model that someone else built, a large language model, one that is good at predicting text. It is trained on a huge corpus of human language, and it is good at answering questions like “write a poem about a frog in the style of W.B. Yeats.” And you ask it questions like “write a report about whether I should buy Nvidia Corp. stock in the style of Warren Buffett.” And then it trains on the writing style of Warren Buffett, which reflects his thinking style, and its answer to your question — you hope — actually reflects what Buffett might say, or what he might say if he was a computer with a lot of time to think about the question. And because Warren Buffett is good at picking stocks, this synthetic version of him is useful to you. You read the report, and if robot Warren Buffett says “buy” you buy.

The first approach makes obvious intuitive sense and roughly describes what various quantitative investment firms actually get up to: There might be patterns in financial data that predict future returns, and deep learning is a statistical technique for finding them.

The second approach seems … sort of insane and wasteful and indirect? Yet also funny and charming? It is an approach to solving the problem by first solving a much harder and more general problem: Instead of “go through a ton of data to see what signals predict whether a stock goes up,” it’s “construct a robot that convincingly mimics human consciousness, and then train that robot to mimic the consciousness of a particular human who is good at picking stocks, and then give the robot some basic data about a stock, and then ask the robot to predict whether the human would predict that the stock will go up.” 

My impression is that there are people using the first approach with significant success — this is roughly, like Renaissance Technologies — and the second approach is mostly me making a joke. But not entirely. The second approach has some critical advantages:

1. Somebody else — OpenAI or xAI or DeepSeek or whoever — already built the large language model for you, at great expense. If you are on the cutting edge of machine learning and can afford to pay for huge quantities of data and researchers and computing capacity, go ahead and build a stock-predicting model, but if you are just, say, an academic, using someone else’s model is probably easier. The large language model companies release their models pretty widely. The stock model companies do not. You can’t, like, pay $20 a month for Rennaissance’s stock price model.
2. Because the large language model’s output is prose, its reasoning is explainable in a way that the stock model is not. The stock model is like “I have looked at every possible combination of 100,000 data time series and constructed a signal that is a nonlinear combination of 37,314 of them, and the signal says Nvidia will go up,” and if you ask why, the model will say “well, the 37,314 data sets.” You just have to trust it. Whereas robot Warren Buffett will write you a nice little report, with reasons you should buy Nvidia. The reasons might be entirely hallucinated, but you can go check. I wrote once: “One criticism that you sometimes see of artificial intelligence in finance is that the computer is a black box that picks stocks for reasons its human users can’t understand: The computer’s reasoning process is opaque, and so you can’t be confident that it is picking stocks for good reasons or due to spurious correlations. Making the computer write you an investment memo solves that problem!”
3. I do think that the aesthetic and social appeal of typing in a little box to have a chat with your friend Robot Warren is different from the black box just giving you a list of stocks to buy. This probably doesn’t matter too much to rigorous quantitative hedge funds, but it must matter to someone. We talked last year about a startup that was launching “a chatbot that offers stock-picking advice” to retail brokerage customers, and it seemed like the goal of the project was not “the chatbot will always tell you stocks that will go up” but rather “the chatbot will offer a convincing simulacrum of talking to a human broker,” who also will not always tell you stocks that will go up. You call the broker anyway. Now you can text the chatbot instead.

And so we also talked last year about an exchange-traded-fund firm that would use large language models to simulate human experts — ones with characteristics of particular humans, like Buffett — to make stock picks. Why use LLMs rather than build a model to directly predict stock prices? Well, because the LLM is already there, and the data is already there, and the schtick is a little more human than “here’s our black box.”

Anyway here’s a paper on “Simulating the Survey of Professional Forecasters,” by Anne Lundgaard Hansen, John Horton, Sophia Kazinnik, Daniela Puzzello and Ali Zarifhonarvar:

Though Levine does a characteristically great job laying out the questions in a clear and insightful way, on at least one point, I think he's not just wrong, but the opposite of right. The LLM may appear to be less opaque, but it is actually the blacker black box.

Normally, when we use the term "black box model," we mean that we know the data that goes in and can see the scored data that comes out, but the process by which it is arrived at is so complex and computation-intensive that we can't say exactly what happened. However, in practice, that's not entirely true. We can analyze the output, identify the main drivers of the model, and flag potential biases and other problems. We can perturb the input data, leaving out certain parts, and observe how the output is affected. In most real-world cases I've seen, you can reverse-engineer the model, creating something remarkably close that uses a manageable and, more importantly, comprehensible dataset and series of calculations. This simpler, reverse-engineered model won't use the same data as the black box, but it will be transparent, will very likely use the same categories of data and generally capture the underlying relationships and sometimes perform almost as well.

I have never done anything related to stock prediction, but I have worked with models predicting consumer behavior, and I'm betting that the underlying process is somewhat similar. Let's take the example of a credit card company building a black-box model to predict which customers are likely to default on their debts. In addition to transaction and payment history, the company has access to a huge amount of data from credit bureaus, vendors such as Acxiom, publicly available information, and macroeconomic data. We're talking about tens of thousands of variables going into that model. It is not possible for a person or even a team to go through all of these fields one by one, but at a more general level, it is possible to know what kind of data is going in and to maintain some standard for quality and relevance.

If your training data is everything that can be scraped from the internet, it is effectively unknowable. In the traditional black-box scenario, we know the data and the output; only the middle part of the process is opaque. With large language models, however, everything before the final answer is shrouded in darkness.

Your training data may include the writings of Warren Buffett, the text of A Random Walk Down Wall Street, and the archives of The Wall Street Journal, but it can also contain horoscopes, blogs from "buy the dip" Robinhood day traders, and market analysis from crypto investors. The style and word choice might resemble those of the Oracle of Omaha, but the underlying ideas might come from the Rich Dad Poor Dad guy.

 

The Grandiosity/Contribution Ratio -- another newly relevant repost

One of the recurring threads at the blog for years now has been the Lords of Ithuvania, the way we have collectively treated people who stumbled upon huge fortunes in the tech industry as super-capable, often almost Messianic figures who can solve any problem and who go unchallenged when making even the most delusional boasts—like claiming they can cure all diseases. That myth is now proving extraordinarily costly.

Tuesday, January 23, 2018

The Grandiosity/Contribution Ratio

From Gizmodo [emphasis added]

Zuck and Priscilla laid out the schematics for this effort on Facebook Live. The plan will be part of the Chan Zuckerberg Initiative and will be called simply “Chan Zuckerberg Science.” The goal, Zuck said, is to “cure, prevent, or manage all diseases in our children’s lifetime.” The project will bring together a bunch of scientists, engineers, doctors, and other experts in an attempt to rid the world of disease.

“We want to dramatically improve every life in [our daughter] Max’s generation and make sure we don’t miss a single soul,” Chan said.

Zuck explained that the Chan Zuckerberg Initiative will work in three ways: bring scientists and engineers together; build tools to “empower” people around the world; and promote a “movement” to fund science globally. The shiny new venture will receive $3 billion in funds over the next decade.
...

“Can we cure prevent or manage all diseases in our children’s lifetime?” Zuck asked at one point. “This is a big goal,” he said soon after, perhaps answering his own question.

Obviously, any time we can get some billionaire to commit hundreds of millions of dollars a year to important basic research, that's a good thing. This money will undoubtedly do a tremendous amount of good and it's difficult to see a major downside.

In terms of the rhetoric, however, it's useful to step back and put this into perspective. In absolute terms $3 billion, even spaced out over a decade, is a great deal of money, but in relative terms is it enough to move us significantly closer to Zuckerberg's "the big goal"? Consider that the annual budget of the NIH alone is around $35 billion. This means that Zuckerberg's initiative is promising to match a little bit less than 1% of NIH funding over the next 10 years.

From a research perspective, this is still a wonderful thing, but from a sociological perspective, it's yet another example of the hype-driven culture of Silicon Valley and what I've been calling the magical heuristics associated with it. Two of the heuristics we've mentioned before were the magic of language and the magic of will. When a billionaire, particularly a tech billionaire, says something obviously, even absurdly exaggerated, the statement is often given more rather than less weight. The unbelievable claims are treated less as descriptions of the world as it is and more incantations to help the billionaires will a new world into existence.

Perhaps the most interesting part of Zuckerberg's language here is that it reminds us just how much the Titans of the Valley have bought into their own bullshit.