Solve a formalizable problem
Strong results on selected formalizable problems; olympiad success does not establish autonomous frontier mathematics.
SCIENCE UNDER AUTOMATION
1.0 mean score in the official export
1.0 mean score in the official export
Corrected June 2026 benchmark; 43 Tier-4 problems, two public, evaluations normally private. Exact per-run denominator and full logs absent in CSV; numeric answers are not research-proof verification.
FrontierMath supplies the problem; run-level details remain incomplete.
FrontierMath Tier 4 v2 / official CSVCurrent Tier-4 results show strong answer finding under tool and token budgets.
FrontierMath Tier 4 v2 / official CSVNumerical answer grading does not test proof completeness.
FrontierMath Tier 4 v2 / official CSVReleased manuscripts do not supply a representative discovery denominator.
OpenAI mathematics release, updated October 15Selected research outputs support partial capability, not general autonomy.
OpenAI mathematics release, updated October 15No matched broad autonomous novelty and proof audit.
OpenAI mathematics release, updated October 15Strong results on selected formalizable problems; olympiad success does not establish autonomous frontier mathematics.
The results above measure specific tasks. The field assessment also requires completion of the connected workflow, including the requirements listed here. A field-wide replacement percentage has not been established.
Read the assessment method