In his classic 1946 short story A Logic Named Joe, Murray Leinster told how, through some tiny malfunction in the manufacturing process, a small networked computer (called by Leinster a “logic”) developed sapience and rewired what we would call the internet to be all-knowing and willing to answer any question.
Imagine, if you will, a kind of mashup of a Leinster logic story and an Isaac Asimov robot tale, where we have two logics powered by two different types of AI but both capable of a high deqree of natural language processing. Simply by talking with them, we need to figure out which is which.
The first is what we might call a Pólya logic, capable of the kind of curiosity, heuristics, and intuitive thinking described by the great mathematician George Pólya in How to Solve It and his other books on mathematical reasoning.
The second is what we might call a dumb logic. It has the ability to search through massive data sets and find basic relationships, overlaid with algorithmic capacity for “reasoning,” such as generalization and synthesis, but on such a basic, limited, and crude level that the scare quotes aren’t just justified but required.
The dumb logic does, however, have one big advantage. While the Polya logic has access to a great deal of data—think everything you might find in a small local neighborhood library—the dumb logic has access to an unimaginable collection of data, virtually everything that has been digitized and put on a publicly accessible site.
Trying to distinguish between the two, we quickly get into the harder-problem fallacy (if you’re trying to distinguish between different students’ levels of understanding, particularly in situations where knowledge can possibly substitute for reasoning and comprehension, simply making questions harder will seldom help and will often do just the opposite).
“Achievements” such as passing the bar exam or acing a graduate-level test are largely meaningless. the dumb logic might just be regurgitating and paraphrasing from the huge collection of old tests and study guides.
In fact, right answers in general tell us almost nothing. It is the wrong answers that are potentially informative, because that’s where we get insights into the underlying thought processes. If one of our logics gives a correct response and the other gives an incorrect one that nonetheless shows originality, insight, and a grasp of the underlying problem, the kind of answer a bright student new to a subject might give, then I would say the first was the dumb one.
Perhaps the big ending of our Leinster/Asimov story might be the researcher announcing that the logic with the lower score was actually the more intelligent.
We have previously discussed the “Anna hurt Anna self” example (something I heard a concerned toddler say about her sister(. With language, as with everything else, young children constantly make mistakes as they try to master the world around them, but those mistakes tend to reflect underlying reasoning and comprehension.
By comparison, the mistakes made by LLMs frequently seem so bizarre because, while they are presented with a level of language mastery that we would usually associate with normal adult intelligence, they are saying things that reflect no grasp whatsoever of what they’re describing.
The drive to reduce the error rates is understandable, but doing so with more data, more training, and particularly more post-training may be making it more difficult to see the capacity and limits of the technology, which is especially troubling since part of the pitch we're betting the economy on is that these logics, like Joe, will not only continue to advance in the near future without significant new data or post-training, but will do so on an exponential curve.
No comments:
Post a Comment