SCIENCE UNDER AUTOMATION
CAD agents with finite-element feedback, v2
Codex GPT-5.5 and Claude Code Opus-4.7 produced no strict-passing artifact in the main first-attempt engineering sweep. The best configuration met about 20% of typed requirements.
What this result supports
The benchmark grades assembled STEP designs through FEA. Feedback improved geometric metrics, which are separate from full engineering validity.
Evidence record: Primary engineering benchmark preprint. Checked: 2026-10-21.
Where this evidence is used
Engineering
- Translate requirements into a coherent design: CAD benchmarks show progress with tool feedback and substantial failures on complete typed requirements.
- Check physical constraints and failure criteria: FEA makes structural checks possible; no strict-passing first-attempt artifact appeared in the cited harder sweep.
- Fabricate and integrate the complete system: The cited design evaluations end with computational artifacts; real integration remains specialist work.
- Resolve failures and accept the deployed system: Automated feedback helps iterate designs, while responsibility for physical validation remains outside the demonstrated agents.