In any case, IMHO I think AI SWE has happened in 3 phases:
Pre-Sonnet 3.7 (Feb 2025): Autocomplete worked.
Sonnet 3.7 to Codex 5.2/Opus 4.5 (Feb 2025-Nov 2025): Agentic coding started working, depending on your problem space, ambition and the model you chose
Post Opus 4.5 (Nov 2025): Agentic coding works in most circumstances
This study was published July 2025. For most of the study timeframe it isn't surprising to me that it was more trouble than it was worth.
But it's different now, so I'm not sure the conclusions are particularly relevant anymore.
As DHH pointed out: AI models are now good enough.
In any case, IMHO I think AI SWE has happened in 3 phases:
Pre-Sonnet 3.7 (Feb 2025): Autocomplete worked.
Sonnet 3.7 to Codex 5.2/Opus 4.5 (Feb 2025-Nov 2025): Agentic coding started working, depending on your problem space, ambition and the model you chose
Post Opus 4.5 (Nov 2025): Agentic coding works in most circumstances
This study was published July 2025. For most of the study timeframe it isn't surprising to me that it was more trouble than it was worth.
But it's different now, so I'm not sure the conclusions are particularly relevant anymore.
As DHH pointed out: AI models are now good enough.