Run a specified analysis
Closest match to the seeded benchmark: objectives and methodological guidance are supplied.
SCIENCE UNDER AUTOMATION
AI in Computational genomics: capabilities and evidence
A large part of the work already happens inside software. The difficult boundary is between executing an analysis and knowing whether it answers the biological question.
What has AI demonstrated?
0.48 artifact-based score
- Test
- 20 study-scale tasks, 138 output artifacts; GPT 5.6 Sol in the cited BixBench3 version.
- Comparison
- 13 models evaluated; no matched human baseline extracted.
- System and version
- BixBench3 · v2, September 2026
- Evidence checked
- 2026-10-20
Scientists supply objectives and methodological guidance. Artifact score is not fraction of studies autonomously completed.
What does the field require?
Analyse an existing study from raw data
- Choose a defensible analysis design
BixBench3 supplies objectives and methodological guidance.
BixBench3 v2 - Execute study-scale analysis
Best reported artifact score is 0.48; artifact success is not workflow success.
BixBench3 v2 - Interpret and reproduce biological results
Artifacts include interpretation; connected reliable completion is not established.
BixBench3 v2
Discover and validate a new genomic mechanism
- Frame a novel biological question
The benchmark supplies the research question.
BixBench3 v2 - Resolve confounding and causal alternatives
No dedicated causal-validation score is available.
BixBench3 v2 - Validate in independent biological data
Independent mechanism validation is outside this benchmark.
BixBench3 v2
Research tasks
Design a new biological study
Question quality, confounding and validation require separate evidence.
Validate a new mechanism
Often crosses from computation into experiments; the field-level physical rating no longer applies.
Can AI replace the whole workflow?
The results above measure specific tasks. The field assessment also requires completion of the connected workflow, including the requirements listed here. A field-wide replacement percentage has not been established.
Read the assessment method