Reproduce and examine an existing quantitative analysis
See the requirement-level assessment and its sources.
SCIENCE UNDER AUTOMATION
AI in Social & behavioural science: capabilities and evidence
Research on people requires measurements of actual behaviour, defensible comparisons and interpretation in context.
Autonomy · up to 4.4 / 10
Model interval: 2.1–4.4. Assessed 21 October 2026.
Two equally weighted workflow families: a bounded operational task and an investigation or deployment beyond that task. Inherited subfields use this parent scenario.
Calculation and work mix
Each requirement uses the operational scale from 0 (independent execution unestablished) to 4 (reliable independent execution in the stated setting). A workflow takes its lowest requirement level. The two workflows receive equal weight; the headline is 1 + 9 × weighted level / 4.
These levels are source-linked model assignments, not measurements reported by the papers. “Up to” is the upper model bound, not a probability or a forecast date.
Reproduce and examine an existing quantitative analysis: 1–2 / 4.
Establish a new result about people or institutions: 0–1 / 4.
Upper bound with 75:25 workflow weights: 4.9. With 25:75 weights: 3.8.
What has AI demonstrated?
Across 221 tasks from 54 papers, Claude Code achieved 93.4% task accuracy and 78.0% paper accuracy; Codex achieved 62.1% and 35.8%.
- Test
- Primary preprint; three independent runs
- Comparison
- Comparators vary by study; see the linked sources.
- System and version
- SocSci-Repro-Bench · 2026-06-09
- Evidence checked
- 2026-10-21
Original replication materials were provided. Ten tasks concerned identifying absent materials. Prompt framing could induce confirmatory specification search.
What does the field require?
Reproduce and examine an existing quantitative analysis
- Run the supplied research code and data
Autonomy level: 2–3 / 4.
SocSci-Repro-Bench demonstrates connected computational reproduction with variable paper-level reliability.
SocSci-Repro-Bench - Identify missing inputs and retrieve further resources
Autonomy level: 1–2 / 4.
Missing-material detection can succeed, but retrieving new replication resources remains difficult.
SocSci-Repro-Bench · ReplicatorBench, v3 - Interpret results without confirmatory distortion
Autonomy level: 1–2 / 4.
Prompt framing affects specification choices; domain checks remain material to interpretation.
SocSci-Repro-Bench
Establish a new result about people or institutions
- Recruit and measure an appropriate human population
Autonomy level: 0–1 / 4.
Simulated respondents are not reliable replacements for representative cross-national survey measurements.
Simulated preferences across 74 countries - Design a causal comparison and address confounding
Autonomy level: 1–2 / 4.
Computational replication supports analysis but does not supply a new field design or intervention.
ReplicatorBench, v3 - Validate interpretation in the relevant social context
Autonomy level: 1–2 / 4.
Cross-country errors and prompt sensitivity require contextual scrutiny.
Simulated preferences across 74 countries · SocSci-Repro-Bench
Research tasks
Establish a new result about people or institutions
See the requirement-level assessment and its sources.
Can AI replace the whole workflow?
The results above measure specific tasks. The field assessment also requires completion of the connected workflow, including the requirements listed here. A field-wide replacement percentage has not been established.
Read the assessment method