SCIENCE UNDER AUTOMATION
ReplicatorBench, v3
Agents were evaluated on retrieving replication resources, executing computational experiments and interpreting results. Resource retrieval remained a major difficulty.
What this result supports
The small benchmark includes replicable and non-replicable claims. Computational replication differs from collecting a new human-subject dataset.
Evidence record: KDD 2026 camera-ready primary report. Checked: 2026-10-21.
Where this evidence is used
Social & behavioural science
- Identify missing inputs and retrieve further resources: Missing-material detection can succeed, but retrieving new replication resources remains difficult.
- Design a causal comparison and address confounding: Computational replication supports analysis but does not supply a new field design or intervention.