Some of today’s most advanced AI models perform surprisingly poorly on a classic psychology experiment known as the Stroop test, which measures attention and cognitive control. In the test, participants must identify the color of a word’s text while ignoring the word itself—for example, saying “blue” when the word “red” is printed in blue ink. While AI models handled short versions of the task well, their performance deteriorated sharply as the tests became longer and more demanding.
Researchers discovered that as task complexity increased, many models began defaulting to reading the word rather than identifying the ink color, causing accuracy to collapse. Some systems reportedly dropped from over 90% accuracy on simple tasks to near failure under more challenging conditions. The findings suggest that the “attention” mechanisms used by transformer-based AI models differ fundamentally from the executive control processes found in the human brain.
The study's authors argue that this weakness could represent a significant obstacle on the path toward artificial general intelligence (AGI). Modern AI systems rely on mathematical self-attention mechanisms that are highly effective for language prediction and pattern recognition, but they may lack the biological attention systems humans use to filter distractions, maintain focus, and adapt to changing goals. The researchers suggest that incorporating more brain-like attention mechanisms may be necessary for achieving truly human-level intelligence.
The results add to a growing debate about whether scaling current AI architectures alone will be enough to reach AGI. While large language models continue to achieve impressive results in coding, reasoning, and content generation, studies like this reveal persistent cognitive limitations beneath their capabilities. Rather than proving that human-level AI is impossible, the research suggests that future breakthroughs may require fundamentally new approaches that go beyond simply making existing models larger and more powerful.
