ChatGPT and the Decline of Accuracy

Shrugging at fallibility

Maybe the most surprising thing about ChatGPT and other LLMs like Google’s Bard and Microsoft’s Bing Chat (formerly Sydney) is that they’re out there in the wild. Taking a step back, look at the situation: we have some of the biggest tech titans on earth producing and promoting LLM-based chatbots that regularly give misinformation, contradict themselves, and produce incoherent nonsense. Amazingly enough, they also frequently produce coherent truths and accurate information. But chatbots are incredibly fallible, and promoting them to users sends a clear message that accuracy is hardly sacred. OpenAI is no more humble with its ChatGPT than the other companies: Quora uses it to answer questions and offers it above human responses.

But as so often happens, we risk localizing the problem. AIs produce novel, amusing, and/or scary mistakes, but the problem is far more holistic. The voluminous human-created linguistic data on which LLMs are trained lacks those errors unique to LLMs but it is littered with errors of the far more traditional sort—far too many for us to ever correct. LLMs and other deep learning AIs need ungodly amounts of training data before they perform passably well, and that amount is so large as to be unscreenable by anything except…another AI.

Wikipedia is one of the major sources of free training data for most chatbots, and for two decades now we’ve heard that Wikipedia simply isn’t reliable. Wikipedia’s own editors stereotype themselves as “Randy from Boise, the archetypal uninformed but relentless Wikipedia editor.” Being human-written, Wikipedia doesn’t descend into nonsense, yet deliberate and accidental errors haunt its pages. While editors wage war over hot-button issues, minutiae of history simply go unchecked and ignored. Yet that’s never stopped Wikipedia from appearing at the top of a plurality of Google and Bing search results, nor being used as fertile training data for LLMs.

So long before ChatGPT, the notion of truth in massive online human-machine networks (meganets, as I call them) was not exactly rigorous. I cite Wikipedia as a convenient and prominent scapegoat, but the reality is that any imaginable large corpus used as training data is going to be (a) filled with a non-trivial amount of mistakes, and (b) too big to have its mistakes fixed before being dumped into a LLM chatbot.

It remains true that even with a pristine, perfect corpus, chatbots as they exist today would still go regularly off-track. My point is that the problem goes far deeper than AI technology and into human and online society itself. We are already settling for a declining notion of what is “good enough” to be taken as true, and that collective decline feeds back onto itself to produce a further decline. Chatbots amplify this trend, as technology so often does, but the internet has already done the heavy lifting.

Ultimately, the problem is size and scale. With the cost of publication having been reduced to near-zero, we have failed to find any filtration mechanisms that reliably percolate “truth” to the top. We produce data and content at exponentially greater rates than we did twenty years ago, which is then fed back into the systems that generate it in an out-of-control feedback loop.

The old mechanisms, it’s true, were far from perfect: elites and gatekeepers imposing their own biases in choosing what was elevated for mass consumption. But to go from that possibility of control to the lack of control means that we are gradually accepting a degraded value of truth. There’s too much to review, too much to consume, too much to adjudicate. As evidence of our increasing acceptance of error, inaccuracy, and simple loss of control, I offer the chatbots.

Sure, when things really count (as in courts of law), there will be controls on the excesses of inaccuracy. But we have already seeded future AIs with so much nonsense that it will only echo and reverberate more and more going forward (I see mitigation as being possible, but nothing like stringency). Not only that, but we’ve already seeded our future human societies with that nonsense.