This article was originally published on The Atlantic.
In addition to being a dismal grocery shopper, ChatGPT has struggled in the past to do basic math. We think of computers as logical and exacting, but ChatGPT is something different: a large language model that has been trained on big chunks of the internet to create associations between words, which it then “speaks” back to you. It may have read the encyclopedia, but it is not itself an encyclopedia. The program is less concerned with things being true or false; instead, it analyzes large amounts of information and provides answers that are highly probable based on our language patterns.
Some stochasticity or randomness—what the computer scientist Stephen Wolfram calls “voodoo”—is built into the model. Rather than always generating results based on what is most likely to be accurate, which would be pretty boring and predictable by definition, ChatGPT will sometimes choose a less obvious bent, something that is associated with the prompt but statistically less likely to come up. It will tell you that the word pours finishes the idiom beginning with “When it rains, it …” But if you push it to come up with other options, it may suggest “When it rains, it drizzles” or “When it rains, it storms.” As Kathleen Creel, a professor of philosophy and computer science at Northeastern University, put it: “When you give it a prompt, it says, Okay, based on this prompt … this word is 60 percent most likely to be a good word to go next, and this word is 20 percent, and this word is 5 percent.” Sometimes that less likely option is inaccurate or problematic in some way, leading to the popular criticism that large language models are “stochastic parrots”: able to piece together words but ignorant of meaning. Any given chatbot’s randomness can be dialed up or dialed down by its creator.