Bridging Generative AI and Truth: Ancient Lessons for Modern Tech
When a seasoned lawyer last year filed a brief citing six precedent-setting cases, he trusted the AI chatbot that assisted him. To his embarrassment, the judge informed him that "six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations." This incident highlights a sobering reality: our most advanced AI systems can confidently produce convincing falsehoods.
In the rapidly evolving world of artificial intelligence, the initial awe inspired by chatbots like ChatGPT is giving way to a more cautious assessment due to incidents like these1 and the growing pile of evidence for “hitting the data wall”2 3 for such models. These large language models, capable of generating human-like text, translating languages, and answering questions with remarkable fluency, have set soaring expectations. Yet, for all their sophistication, these models suffer from a fundamental flaw: they "hallucinate".
Understanding the Roots of AI Hallucinations
These AI systems learn by analyzing vast amounts of text data from the internet, predicting the next word in a sentence based on patterns they've observed, not on verified facts. They build associations between words that are popular or frequently paired, regardless of their truthfulness. While developers attempt to correct these tendencies by training models additionally on curated and human-labeled data, this refined dataset is often very small compared to the unfiltered ocean of text they initially consume. And to make things worse, we’ve used up nearly all the public text data and are in dire need of “a few additional internets”.
A team from Stanford University4 recently evaluated several legal AI systems (some even advertised as hallucination-free) and discovered a spectrum of inaccuracies. These ranged from outright false statements to invalid references to legal precedents. For experts, this is not surprising since the inclination to generate diverse text is built into the very nature of these models. It's a feature of their design, not a bug.
Learning from the Past
This modern technological phenomenon resembles a similar challenge in our history. In ancient Greece, people once sought wisdom from oracles who often wove convincing tales to interpret dreams or used popular consensus —often influenced heavily by the tricks of rhetoric— as a substitute for objective truth5. It wasn't until philosophers like Aristotle introduced principles of formal logic that society began to systematically verify claims and distinguish between ordinary statements and reality.
Much like those ancient times, we find ourselves amazed by the seemingly prophetic outputs of AI chatbots, yet anxious about their reliability. The parallels are striking: both scenarios involve reliance on sources that offer impressive but unverified information, highlighting the timeless intellectual struggle to separate truth from fiction.
Diverging Perspectives in AI Development
Within the AI community, there's an ongoing debate on how to address the problem of hallucinations. One line of thought believes that scaling up is the key, arguing that by increasing computational power, expanding model parameters, and training on even larger datasets with higher-quality labels, language models will eventually learn to avoid hallucinations. They view current shortcomings as temporary hurdles that will diminish as we throw more resources in the same manner.
Conversely, the second camp believes that simply making models bigger won't solve the problem and supports a fundamental shift away from the "scale-is-all-you-need" approach. This perspective calls for developing new architectures and algorithms with other breakthroughs that can systematically distinguish fact from fiction, with the current models serving perhaps as tools for the next generation of AI systems.
As we stand at this crossroads in the history of AI, we can both acknowledge the effectiveness of current generative AI models and leverage them to their full potential while also learning from the intellectual journey of our own species. Before making bold claims about artificial general intelligence systems seemingly around the corner, we must first understand and acknowledge the shortcomings of today's AI models. Only by subjecting our AI dreams to the harsh light of reason can we hope to create models that rise above mere hallucinations.
Ilya Sutskever NeurIPS 20025 Test of Time Award talk
The Wall Street Journal, The Next Great Leap in AI Is Behind Schedule and Crazy Expensive
I agree that finding a definition for the “objective truth” is not an easy task and we have created a whole field of epistemology, but nonetheless we can assume a working definition that allows us to converse, even if we never achieve absolute certainty.