&imwidth=800&imheight=600&format=webp&quality=medium "Four AI models fail Math test they could not cheat on, humans score a perfect 10")
Artificial Intelligence systems might have solved an 80-year-old Math problem, but a test designed to evaluate it against top-tier academicians has revealed that it still has a long way to go when it comes to new and complex research-level mathematics. Four AI systems failed the test, which is part of "First Proof," a rigorous new AI benchmarking mechanism. The AI systems were given 10 research-level maths problems, and their answers were assessed by a jury of anonymous human specialists. To prevent AI from simply scanning through its training data, all the questions were new and had never been published anywhere. They were provided by 10 researchers from a broad range of mathematical specialities who had solved them at some point in their careers. A group of 30 mathematicians checked the answers, and the results proved that AI does not come close to human mathematicians in novel problems. The findings were published in the Nature journal.
Also Read: Company accidentally blows away Rs 4,800 crore on Claude AI in a month because of a tiny oversight
The First Proof team carried out the test in an entirely autonomous way without humans being involved in any way. The AI systems also had to be publicly available. This ruled out Google’s Aletheia, which has been designed specifically for solving maths problems, and the full, unreleased version of Claude Mythos. OpenAI entered its ChatGPT 5.5 Pro model. Three academic groups provided the rest of the models - from the University of California, Los Angeles (UCLA); Princeton University in New Jersey; and the Swiss Federal Institute of Technology (ETH) in Zurich. They built "harnesses"—automated systems that prompted existing chatbots, such as ChatGPT, Gemini, and Claude, to repeatedly check the answers.
AI vs human mathematicians: The results
The top-performing AI system was able to solve only 6 out of 10 problems. Since all of them had been solved by mathematicians, this proved that human expertise in the field still outperforms AI on unseen, highly rigorous research problems. ETH Zurich stood first, whose harness had used ChatGPT to generate answers. UCLA was second (ChatGPT harness), followed by OpenAI (standalone ChatGPT 5.5 Pro), and Princeton (using a Gemini 3.1 Pro harness).
Trending Stories
)
Claude AI down? Anthropic confirms outage as users face errors across chatbot and API
)
Missile lands in Poland, leaves crater as PM Donald Tusk says 'all indications' point to Moscow
)
23ft python swallows man returning from party in Indonesia; family find body whole inside reptile
)
Ramayana trailer X review: Ranbir Kapoor, Yash shine in a visual spectacle fans call 'cinema at its finest'
)
India avoids higher US tariff, secures lower 10% rate. Here's how
&im=FitAndFill=(700,400))
Why are iPhones so expensive compared to Android phones?
)
Assam floods 2026: Salman Khan, Alia Bhatt and other Bollywood celebrities extend support to affected families
)
Anthropic reveals India is Claude AI's No. 2 market. But what are Indians actually using the AI for? | WION Decodes
)
3.6mn people track 24hr Qantas flight from Melbourne to Toulouse: Pilot shifts, fuel tank and other details
)
The Royals enter the Doctor Who universe: King Charles and Queen Camilla beam into Commonwealth Games in TARDIS
)
Are you an Indian on an H-1B visa? New US bill could make getting Green Card easier
)
NEET Protests riddled with misinformation? Delhi Police blocks 480 Pakistan-based handles spreading 'fake news' since Operation Sindoor
)
Russia-Ukraine war: Putin mobilises more troops for Ukraine's war decisive invasion
)
British Airways flight comes close to 'nosediving' over London
)
28-year-old woman PhD scholar dies of electrocution on Panjab University campus
)
This Ramayana actor took ZERO fee from Rs 4000 crore film. Guess who?
)
582 tonnes, 150 million°C, 60 years: WION Decodes the mind-boggling numbers behind China's 'Artificial Sun'
)
Student denied admission to IIT Kanpur's cybersecurity program hacks websites to prove a point
)
Lift accident? Maintenance, builders & makers are ALL liable now, rules Supreme Court: 3 major regulations | WION Decodes
)
Delhi Consumer Court fines Indian Railways Rs 25,000 for making RAC passengers share one blanket
)
Ramayana Part 1 trailer out! Ranbir Kapoor, Sai Pallavi and Yash lead Nitesh Tiwari’s epic saga
)
Indian forces used pellet guns on Jul 20? SC hearing, police diary entry, MLC report and injured protesters' claims
)
Netanyahu spent thousands of dollars on garden maintenance in wartime, report says | Gravitas
)
Brewery offering 'free beer' the day Trump dies has license revoked - 'Will have to pry me out of my taprooms'
)
Trump says Iranian school bombing photos could be AI-generated - 'I don’t think...'
)
Kangana Ranaut clarifies her 'Generation Gutter' remark: 'Gen Z of our country are taking us to such heights' - watch
)
CJP founder Abhijeet Dipke returns to Maharashtra, receives grand welcome in hometown
)
Mysterious projectile leaves massive crater in Poland near Ukraine border, probe launched
)
Commonwealth Games 2026 Day 8: India's full schedule, medal events and match timings
)
Nearly Rs 29 crore cash, 15 kg gold found at ex-bus driver's house in West Bengal
The results show that AI models have a long way to go before they can function autonomously as reliable research assistants, proof-checkers, and problem-solvers for human mathematicians. First Proof will come up with more versions of the test in future to correctly decide when this could actually become possible.
