A recent study published in the BMJ reveals that older large language models (LLMs), including chatbots, exhibit cognitive decline similar to humans. Researchers found that these AI models, commonly used for tasks like medical diagnostics, show deteriorated abilities in crucial areas over time. This decline could impact their effectiveness in providing reliable medical diagnoses.
Study Methodology
The study evaluated several AI models—OpenAI’s ChatGPT, Anthropic’s Sonnet, and Alphabet’s Gemini—using the Montreal Cognitive Assessment (MoCA).
AI Test Results
The latest version of ChatGPT (version 4) scored 26 out of 30, indicating it met the passing threshold for cognitive ability. However, older models like Gemini 1.0 scored only 16, revealing significant impairment. These results suggest that as AI models age and remain in use, they may experience a form of cognitive decline.
While the LLMs performed well in areas such as language, attention, and abstraction, they struggled with visual-spatial tasks and executive functions. Delayed recall—a critical metric for assessing memory retention—was also a common area of weakness.
Read: SPHEREx Space Telescope to Search for Life’s Ingredients
Implications for Medical Diagnoses
AI-driven tools are increasingly relied upon for medical diagnoses due to their speed and accuracy in identifying anomalies in medical data. However, the study raises doubts about whether older models are reliable for this purpose. “These findings challenge the assumption that artificial intelligence will soon replace human doctors,” the study authors wrote. They warned that cognitive decline in LLMs could erode patient confidence in AI-driven healthcare tools.
Why Cognitive Decline Occurs
Cognitive decline in AI may stem from outdated training data, lack of updates, or limitations in older algorithms. As newer models are developed, they typically receive continuous training on larger and more diverse datasets. In contrast, older models may stagnate, leading to reduced effectiveness over time.
Moving Forward
The study highlights the importance of regular updates and improvements for AI models, particularly those used in critical fields like healthcare. Developers may need to implement systems that monitor AI performance over time and replace or update models as needed to prevent decline.
Ultimately, while AI holds great promise for medical and other high-stakes applications, human oversight remains essential to ensure accuracy and reliability.
Follow us on Google News, Instagram, YouTube, Facebook,Whats App, and TikTok for latest updates
