The Tortoise Wins Again: When AI Coding Tools Make Developers Slower

In July 2025, a randomised controlled trial found that experienced open-source developers took 19% longer to complete tasks when AI tools were allowed. The same developers believed the tools had made them 20% faster.

The METR Study

METR (Model Evaluation & Threat Research) published “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” on July 10, 2025. The authors were Joel Becker, Nate Rush, Beth Barnes and David Rein.

• Participants: 16 experienced open-source developers.
• Tasks: 246 real issues, including bug fixes, features and refactors; 136 allowed AI and 110 did not.
• Task length: about 2 hours on average.
• Repositories: mature projects averaging about 1.1 million lines of code.
• Experience: developers had an average of 5 years and about 1,500 commits on their specific repository.
• Tools: mainly Cursor Pro with Claude 3.5 Sonnet and Claude 3.7 Sonnet.
• Pay: $150 per hour.
• Data collection: February to June 2025.

Expectation vs Result

• Developer forecast before tasks: 24% speedup.
• Developer estimate after tasks: 20% speedup.
• Economics experts’ forecast: 39% speedup.
• Machine learning experts’ forecast: 38% speedup.
• Measured result: 19% slowdown, with a 95% confidence interval of +2% to +39%.

Where the Time Went

• Developers accepted fewer than 44% of the code suggestions generated by Cursor.
• About 9% of AI-allowed working time went to reviewing and cleaning AI output.
• METR examined 20 possible contributing factors and found evidence that 5 likely contributed to the slowdown.
• Candidate factors included the learning curve of Cursor and the high quality standards and implicit requirements of mature repositories.

The 2026 Follow-Up

On February 24, 2026, METR published an update on a new study that began in August 2025:

• Scale: 57 developers, 143 repositories and more than 800 tasks.
• Returning developers (10): task time changed by −18%, with a confidence interval of −38% to +9%.
• New developers (47): task time changed by −4%, with a confidence interval of −15% to +9%.
• Significance: both intervals include zero, so neither result is statistically significant.
• Selection effect: 30% to 50% of surveyed developers reported avoiding tasks they did not want to do without AI.

METR described the late-2025 data as “only very weak evidence” and stated that the true speedup is probably higher than the estimates.

Learning Costs

A randomised trial published by Anthropic on January 29, 2026, tested 52 mostly junior engineers learning the Python async library Trio:

• The AI-assisted group scored 50% on a follow-up quiz.
• The hand-coding group scored 67%, a 17-point gap (p = 0.01).
• The AI group finished about 2 minutes faster, a difference that was not statistically significant.

What Developer Surveys Report

Stack Overflow Developer Survey 2025:

• 84% of respondents use AI tools.
• 46% distrust the accuracy of AI tools, up from 31% in 2024.
• Only 3% “highly trust” AI output.
• 66% are frustrated by AI answers that are “almost right, but not quite”.
• 45% say debugging AI-generated code takes longer.
• Favourable views of AI tools fell to 60%, from 72% in 2024 and 77% in 2023.

Google DORA reports:

• 2024: every 25% increase in AI adoption was associated with a 1.5% drop in delivery throughput and a 7.2% drop in delivery stability.
• 2025: 90% of nearly 5,000 surveyed professionals use AI, with a median of 2 hours per day.
• 2025: more than 80% report higher productivity, while AI adoption is still associated with lower release stability.
• Trust in 2025: 24% trust AI “a great deal” or “a lot”; 30% trust it “a little” or “not at all”.

DORA lead Nathen Harvey stated: “100% trust in AI would be wrong.”

Productivity at the Team Level

Developer productivity data affects hiring decisions. Employers increasingly favour staff who can combine AI tools with domain expertise, which is why the AI-savvy coworker may pose a bigger risk to jobs than AI itself. The METR data shows that the advantage depends on context: mature codebases with strict standards reduced the benefit of AI suggestions.

Developer Tool Domains

Many AI coding tools and developer platforms operate on the .io extension, the country-code top-level domain of the British Indian Ocean Territory. Startups that buy io domain names use the two-letter suffix because “I/O” is the standard abbreviation for input/output in computing.