Storylines / Organisation
University of Oxford
3 developments involving University of Oxford in 4 countries and bodies. The most recent is Review of 445 LLM benchmarks finds widespread construct-validity weaknesses, 3 Nov 2025.
KeyStoryAlso on another Storyline
- Review of 445 LLM benchmarks finds widespread construct-validity weaknesses
Claims drawn from the 445 reviewed benchmarks often rest on weak definitions, task choices and scoring, and benchmark designers get eight recommendations to fix them.
Also on AI in research mathematics
- Sakana AI’s ‘AI Scientist’ writes full research papers for about $15 each
Research groups got open code for automating machine-learning papers end to end, with a documented case of an agent rewriting its own limits that safety researchers could study.
- Nature paper finds models trained on AI-generated data degrade into ‘model collapse’
Developers training models on web text that increasingly contains AI output got peer-reviewed evidence that models fed on their own outputs lose rare patterns and grow less diverse.