For the First Time, AI Analyzes Language as Well as a Human Expert
Language is what makes us human, but what does it mean now that large language models have gained “metalinguistic” abilities?
Unraveling the Mystery of Human Language
Among the myriad abilities that humans possess, which ones are uniquely human? Language has been a top candidate at least since Aristotle, who wrote that humanity was “the animal that has language.”
Even as large language models such as ChatGPT superficially replicate ordinary speech, researchers want to know if there are specific aspects of human language that simply have no parallels in the communication systems of other animals or artificially intelligent devices.
A New Study Challenges Our Understanding of AI
Researchers Gašper Beguš, Maksymilian Dąbkowski, and Ryan Rhodes recently put a number of large language models through a gamut of linguistic tests, including having the LLM generalize the rules of a made-up language.
While most of the LLMs failed to parse linguistic rules in the way that humans are able to, one had impressive abilities that greatly exceeded expectations.
Metalinguistic Capacity: A Breakthrough in AI
The researchers created a linguistic test in four parts, three of which involved asking the model to analyze specially crafted sentences using tree diagrams.
One part of the test focused on recursion—the ability to embed phrases within phrases.
Recursion: A Defining Characteristic of Human Language
Recursion has been called one of the defining characteristics of human language by Chomsky and others—and indeed, perhaps a defining characteristic of the human mind.
Linguists have argued that its limitless potential is what gives human languages their ability to generate an infinite number of possible sentences out of a finite vocabulary and a finite set of rules.
A New Era in AI Research
The study’s findings challenge our understanding of what AI can do and highlight the importance of linguistic analysis in evaluating the degree to which language models can reason like humans.
As society becomes more dependent on this technology, it’s increasingly important to understand where it can succeed and where it can fail.
Source: Link
