Latest
  1. Johns Hopkins Bayview tells 3,182 patients an employee put their health data into a public AI tool
  2. China’s AI compute access expected to double in 2027, with US-designed chips about 65 per cent
  3. UK commits £100m to national AI Biosecurity Lab
  4. Moonshot AI releases Kimi K3.1 Pro with 1-million-token context window
  5. Residents in Newport and Cardiff oppose hyperscale datacentres near homes
  6. Study: Grok 4.7 picked Republicans in 92% of forced midterm answers, GPT-6.1 Sol picked Democrats in 97%
  7. Attackers exploiting the PHP flaws hand the break-in to AI agents, Unit 42 says
  8. Elvis Presley’s rightsholders sue Anthropic over training on 274 songs

Latest / Science and research

ReportScienceFrontierUS InternationalConfirmed

Top AI models solve under 2% of Epoch AI’s new FrontierMath research problems

See how this story connects

Source: arXiv (Epoch AI)Also reported by Epoch AI (OpenAI funding disclosure)

Epoch AI released FrontierMath on arXiv on 7 November 2024, a benchmark of hundreds of new, unpublished mathematics problems written and checked by expert mathematicians. Leading AI models solved under 2% of them at release. In January 2025 Epoch disclosed that OpenAI had commissioned and funded 300 of the problems and owns them, with a 50-problem holdout set whose solutions OpenAI does not see.

Why it matters

AI labs gained an unpublished research-level maths test for reasoning models; in January 2025 Epoch disclosed that OpenAI funded 300 of the problems and sees the solutions to all but a 50-problem holdout set.

When
7 Nov 2024
ID
WTA-N-2024-0031
Who
Epoch AI, OpenAI

Summary by wheresthe.ai, from arXiv (Epoch AI) and the reporting credited above. Checked against the source on 9 Oct 2026.

Read the original at arXiv (Epoch AI)

Source copy: download the page as saved on 18 Oct 2026 (PDF). Also held by the Internet Archive.

Next stop on this Storyline: What happened nextAnthropic says Claude now leads 26% of its AI research and development

Cite this story

WTA-N-2024-0031: “Top AI models solve under 2% of Epoch AI’s new FrontierMath research problems.” wheresthe.ai, 7 Nov 2024. https://wheresthe.ai/d/epoch-frontiermath-benchmark-2024-11/ (source: arXiv (Epoch AI)).

Spotted an error? Email [email protected].