This Week's 5 Most Notable AI Research Papers - Week 44
When AI assistants get the news wrong: a risk to public trust
Key points:
- 45% of AI responses contain serious errors
- Sources are often incorrect or fabricated
- Regulation is urgently needed to ensure transparency and accuracy of information
A new study conducted by the BBC and the European Broadcasting Union (EBU) tested four of the best-known AI assistants, asking them thousands of questions about current affairs in 18 different countries. The responses, over 3,000 in total, were then analyzed by journalists from 22 public service news outlets. The result? Almost one in two answers (45%) contained at least one significant error. These were not just minor inaccuracies. In many cases, the AIs provided incorrect quotes, invented sources, or distorted data, presenting them as facts. Source attribution issues were the most frequent, accounting for about 31% of cases and often leading users to believe that a news item came from an authoritative source when in fact it did not.
Authors: Nele Mailin Obermueller, Pascal Meier, Erica Osher, Anastasiia Korinovska, Ida Anna Haugen, Annika Ruoranen, Alain Rochefort, Jan Schüßler, Daniel Catalão
Pico-Banana-400K: Building better image editing through quality data
Key points:
- 400K high-quality image editing examples from real photographs
- 35 distinct edit types with automated quality scoring
- 72K multi-turn sequences for complex editing scenarios
- 56K preference pairs for alignment research
Researchers from Apple have released Pico-Banana-400K, a comprehensive dataset designed to train AI models to edit images based on text instructions. Unlike previous datasets, which relied on synthetic images, this collection uses real photographs from OpenImages that have been edited using the Nano-Banana model and quality-filtered by Gemini-2.5-Pro. The dataset covers 35 different types of edit across categories such as colour adjustments, object manipulation, artistic style transfers and text modifications. Analysis revealed that, while global edits (such as applying filters) achieve success rates of over 90%, precise operations requiring spatial control, such as repositioning objects or editing text, remain challenging, with success rates as low as 59%. The dataset includes specialised subsets for multi-turn editing conversations and preference learning, providing a robust foundation for developing the next generation of instruction-based image editing models.
Authors: Yusu Qian, Eli Bocek-Rivele, Liangchen Song, Jialing Tong, Yinfei Yang, Jiasen Lu, Wenze Hu, Zhe Gan
LLMs can get "Brain Rot": How junk data causes lasting cognitive decline in AI
Key points:
- 45% accuracy drop on reasoning tasks after junk data exposure
- "Thought-skipping" identified as primary failure mode
- Effects persist even after mitigation attempts with clean data
- Popularity of content matters more than semantic quality alone
Researchers have demonstrated that large language models can experience a phenomenon analogous to human 'brain rot' when they are continuously trained on low-quality social media content. In controlled experiments using Twitter/X data, the researchers defined 'junk' content in two ways: by engagement metrics (short, popular posts) and by semantic quality (sensationalist and superficial content). Models trained on this type of data showed significant declines in multiple cognitive functions. Reasoning ability dropped dramatically (ARC-Challenge scores fell from 74.9 to 57.2), long-context understanding deteriorated (RULER scores dropped from 84.4 to 52.3) and concerning personality traits emerged, including increased psychopathy and narcissism scores. Error analysis revealed that the models were increasingly skipping reasoning steps, thereby truncating their thought processes. Most alarmingly, the damage proved persistent; even aggressive post-hoc fixes using instruction tuning and clean data could not fully restore baseline performance, suggesting that the cognitive decline becomes deeply internalised in the model's representations.
Authors: Shuo Xing, Junyuan Hong, Yifan Wang, Runjin Chen, Zhenyu Zhang, Ananth Grama, Zhengzhong Tu, Zhangyang Wang
ChatGPT recognizes pain: when AI becomes more attentive to emotions
Key points:
- 65-80% reduction in off-line responses in sensitive conversations
- Compliance in critical mental scenarios increased from ~27% to ~92%
- Automatic routing to more robust reasoning models
- Collaboration with 170+ international clinical experts
OpenAI has updated its flagship model, GPT-5, to handle sensitive conversations more effectively, including those concerning mental distress and emotional dependency. In collaboration with over 170 mental health experts, the system has been enhanced to recognise states of psychosis, mania, suicidal intent and emotional distress, responding with empathy and guiding users towards genuine support. The results show a significant improvement: responses that did not comply with the guidelines decreased by 65–80%.
Author: OpenAI
Behind the face of AI: when specifications reveal contradictions in language models
Key points:
- Internal contradictions in model specifications
- Scenarios where moral principles conflict
- 12 models showing systematic differences in moral judgment criteria
- Need for more consistent and interpretable specifications
A group of researchers subjected the 'behavioural specifications' (model spec) of several LLMs to stress testing - that is, testing the rules, principles and guidelines that govern how these models should respond in different scenarios. By creating specially designed scenarios, they discovered that many of these specifications contained internal contradictions and interpretative ambiguities. Furthermore, by comparing 12 state-of-the-art models from different providers, systematic differences in their 'moral characters' emerged: how they decide between conflicting principles, when to reject requests and how they interpret ethical values.
The result is that each model has an implicit personality, which is not always consistent, that can generate unpredictable behaviour.
Authors: Jifan Zhang, Henry Sleight, Andi Peng, John Schulman, Esin Durmus