Something you won't hear me say often (because, as previously mentioned, I despise The New York Times), but this recent op-ed is essential reading. It's essential because computer scientist Cal Newport is one of the most thoughtful and clear-eyed voices in the AI debate.
Anthropic recently dropped a classic of the form: a scary-sounding report titled “When A.I. Builds Itself” that claims A.I. could be moving closer to the capability of “autonomously designing and developing its own successor.” The company hopes that this recursive self-improvement will bring “enormous good” to the world, but also openly worries it might lead to humans “losing control” of these systems.
The public reaction to this report focused on a section that seemed to call for a worldwide pause on A.I. development. But if you read more carefully, it becomes clear that a pause isn’t actually what Anthropic proposes. Its report says that “if it were possible” to slow down the technology, then we should, but so long as the “least cautious” actors were advancing full speed, it suggests that Anthropic will have no choice but to do the same....
Let’s call this strategy “doom trolling.” It’s one of the defining and most arresting properties of our current A.I. moment, and I’ve come to believe that it’s morally indefensible.
There are really only two options for the intentions of A.I. companies when they engage in doom trolling. The first is that they actually believe that the systems they’re building have a nontrivial chance of producing hugely disruptive events — from destroying the economy in the best case to wiping out our species in the worst. If this were true, every reasonable ethical system would argue that there is only one acceptable response: to immediately stop working on any product that might accelerate such a future, and lobby with all of your resources to help force other A.I. companies to do the same. From a moral perspective, any other reaction would be monstrous.
The second option is that these A.I. companies aren’t really concerned about these risks, and that they’re injecting these doses of unresolvable doom for other reasons. They might want to amplify the perceived power of their technology at a time when they’re setting up their initial public offerings. Or they hope their performative reports and somber interviews will help them compete for top engineering talent coming from a Silicon Valley culture that’s steeped in this type of doomerism. The venture capitalist and A.I. adviser David Sacks recently suggested that Anthropic was using fear-mongering tactics as a method of “regulatory capture,” which can impede upstart competitors. Any of these reasons would mean that these companies are laundering the anxiety of millions to improve the financial fortunes of a vanishingly small number of major stockholders. This cynicism would be equally monstrous.When it comes to A.I., we’ve become so used to this tone of helpless, stress-inducing prognostication that we’ve lost sight of its strangeness. Imagine if Ford put out a report saying that it feared its popular F-150 trucks might soon start bursting into flames, but that there was nothing the company could do about it because automotive technology was too inevitable and important to slow down. You’re probably struggling to picture this scenario because no reasonable consumer product company would ever act like this.
The A.I. companies could start behaving the same way. To do so would require that they stop treating A.I. like some inevitable force that they’re struggling to steward. It’s not. It’s a collection of specific tools that these companies are choosing to design and sell according to specific business plans. Accordingly, they need to talk about their offerings like any other consumer product. This means explaining clearly whom these products are for, justifying their benefits and, critically, taking full responsibility for any harm they might cause. Just because A.I. currently enjoys a high-tech sheen doesn’t make it exceptional with respect to common-sense safety standards.
If these A.I. companies insist on continuing to pretend that they’re merely stoic observers of an unavoidable dystopian future, then perhaps it’s time to force the issue. As consumers, we can refuse to play the doom-trolling game. Next time Anthropic releases a dire report, or Sam Altman’s voice cracks as he imagines the disruption that OpenAI is unleashing, we can pivot back to the pragmatic: “OK, but what benefits am I getting by spending $1,000 a month on tokens?” If they continue to ratchet up the doom, then perhaps it’s time to transform dread into ridicule: The earnest pseudoscience of Anthropic’s white papers already borders on satire. The aura surrounding A.I. encourages a fretful submission to these tech leaders, but this could rapidly change.
A few days after the NYT op-ed appeared, Newport made an appearance
on the podcast Better Offline, where he explored the same topics (he has an especially good time putting on his professorial hat and tearing apart Anthropic pseudo-academic white papers) and
put those ideas in the context of the profoundly dysfunctional culture of
Silicon Valley. We will be coming back to this one.


