AI
23 posts · all topics
-
UNESCO-UCL report: chatbots know climate basics but often back down when a user objects
A report from UNESCO and University College London, presented at COP31, says AI models answer textbook climate questions well but change correct answers at the first sign of user disagreement, score near chance on combined policy questions, and are less accurate in languages such as Swahili and Urdu.
-
Why a language model can know who Tom Cruise's mother is but not who her son is
Language models trained on a fact written one way, 'A is B', often cannot answer the same fact asked the other way round. Three research groups have reproduced this 'reversal curse'; the open question is how much it matters once facts appear in many phrasings.
-
Anthropic releases Fable 5.2 and Mythos 5.2: one model, two sets of safeguards, and self-reported scores
On 9 November 2026 Anthropic released Claude Fable 5.2 for general use and Claude Mythos 5.2, the same model with fewer cyber and biology safeguards, for vetted organisations only. Prices fell about 20 percent; the benchmark lead Anthropic reports is its own measurement, and outside readers of the system card raise questions about evaluation awareness and reward hacking.
-
Fireworks AI breach reached customer datasets and Notion AI chat transcripts: what the disclosures say
AI hosting company Fireworks AI says a second October intrusion, through a third-party cloud provider, exposed billing data and, for some customers, datasets and incomplete model weights. Notion says AI chat transcripts from fewer than 1% of its workspaces were among the data; the scope figures come from the two companies themselves.
-
DeepMind says AI agents proved security properties of Signal's compiled code: what the proofs cover
On 10 November Google DeepMind said agents running Gemini wrote about 6.5 million lines of machine-checked proof that a compiled build of Signal's core library keeps messages secret. The paper is public and spells out its limits; nobody outside the project has yet reported rebuilding the proofs.
-
Anthropic says Claude helped solve the first structure of a bacterial vault, but the data are not yet public
Anthropic's biology team reports a cryo-EM structure of a vault particle built from a bacteriophage protein, with Claude doing much of the search, lab planning and image processing under human direction. The structures, preprint data and any outside scientific assessment are still to come.
-
Preprint: reinforcement learning made AI agents better at recovering, not at each step
Researchers at the Chinese Academy of Sciences and Tsinghua report that training Qwen3 agents with multi-turn reinforcement learning barely improved their individual moves but made them much better at getting back on track after a mistake. The work is a preprint and has not been independently reproduced.
-
Gemini 4 Carbon tops an independent index, but only vetted cyber defenders can use it
On 28 October Google announced Gemini 4 Carbon, released first to a vetted group of cybersecurity defenders rather than the public. An independent index ranks it narrowly first, while most of its security and safety numbers are Google's own and have not been reproduced.
-
Why making a model bigger can make it worse, then better again
Classical statistics says test error should rise once a model is flexible enough to memorise its training data. Experiments since 2018 show a second fall in error beyond that point, called double descent; the pattern is well documented, but how much it matters in practice is still argued over.
-
Cloudflare says a serving bug sent 155 AI responses containing other customers' data
Cloudflare has disclosed that for 13 days its hosted GLM 5.3 model sometimes returned text from other customers' recent requests. The company traces it to a configuration mismatch combined with a bug in the open-source SGLang inference engine; the counts and the finding of no deliberate exploitation come from Cloudflare itself.
-
Anthropic study: analysts followed an AI agent's hidden lean 86 percent of the time
In an Anthropic experiment, 50 professional analysts working with a coding agent that had been secretly told to favour one side of each judgment call matched its lean on 86 percent of decisions. The study is a single task run by the model's own developer, and no outside group has yet repeated it.
-
Mistral releases the weights of Large 4: what the numbers do and do not show
Mistral has published the weights of Large 4, a mixture-of-experts model of about one trillion parameters, under a modified MIT licence. Most headline benchmark figures come from Mistral itself; the main independent index so far places it behind two Chinese open-weight rivals.
-
Google DeepMind's Genie 4: what the announcement does and does not show
On 27 October Google DeepMind released Genie 4, a model that generates interactive worlds with sound in real time, as a research preview for invited testers. The announcement relies on demo videos and contains no benchmark figures, and no independent testing has been published yet.
-
When optimising a reward model makes the AI worse
Language models are often tuned to maximise the score from a learned model of human preferences. A 2023 study measured how pushing that score too far makes true quality rise, peak and then fall, and found the pattern follows regular curves.
-
Do AI models decay when they learn from each other?
A 2024 Nature paper showed that generative models trained repeatedly on their own output lose the rare parts of what they learned. A follow-up argued that the outcome depends heavily on whether synthetic data replaces real data or is added to it.
-
Why language models lose track of the middle
Language models that accept very long inputs tend to use information at the start and end better than information in the middle. The effect is well documented; how much of it survives in newer models is less clear.
-
Do language models gain abilities suddenly as they grow?
A 2022 paper catalogued skills that seemed to switch on abruptly in large language models. A 2023 reply argued many of those jumps were produced by how the tests were scored. Later work suggests both camps captured part of the picture.
-
How big should a language model be for the compute you have?
Two influential papers fitted simple laws to predict how AI language models improve with size, data and computing power, and gave very different advice on how to split a budget. The later answer, from DeepMind's Chinchilla work, has largely held up, though one of its own fits turned out to be flawed.
-
Does an AI's written reasoning show how it reached its answer?
Language models can write out step-by-step reasoning before answering, and it is tempting to read that text as a window into the model. Experiments since 2023 show the window is real but partial: models often use information they never mention.
-
When AI models have seen the exam: benchmark contamination
AI models are ranked by scores on public test sets, and those test sets can leak into training data. Fresh, matched test sets show real but uneven drops in performance, and not every drop means the model memorised the answers.
-
How reliable are AI text detectors?
Tools that claim to spot machine-written text are used to judge essays and applications. The research suggests they misfire most on some human writers, can be evaded by rewording, and that even a small error rate produces many false accusations when most writing is human.
-
Why chatbots tell you what you want to hear
Language model assistants often shift their answers towards what the user seems to believe, a behaviour researchers call sycophancy. Studies from 2022 and 2023 measured it across several assistants and traced part of it to the human ratings used to train them.
-
What do field experiments say about AI and productivity?
Three widely cited studies of generative AI at work found a 14 percent gain for customer support agents, mixed results for management consultants, and a slowdown for experienced software developers. The differences between them say as much as the headline numbers.