Study: AI coding tools slow down experienced developers by 19%

A new study by AI research nonprofit METR has found that artificial intelligence coding tools actually slowed down experienced software developers by 19% when working on familiar codebases, contrary to the developers’ expectations of a 24% speed improvement. The findings challenge widespread assumptions about AI’s productivity benefits for skilled engineers and raise questions about the substantial investment flowing into AI-powered development tools.

What you should know: The study tracked seasoned developers using Cursor, a popular AI coding assistant, on open-source projects they knew well.

• Before the study, developers expected AI to decrease their task completion time by 24%.
• Even after completing tasks with AI assistance, developers still believed they had reduced completion time by 20%.
• In reality, using AI increased task completion time by 19%, shocking even the study’s authors who had anticipated significant speed improvements.

Why the slowdown occurred: Developers spent considerable time reviewing and correcting AI-generated suggestions that were often directionally correct but not precisely what was needed.

• “When we watched the videos, we found that the AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what’s needed,” said lead author Joel Becker.
• The time spent on corrections outweighed any initial productivity gains from AI assistance.

The big picture: These results contradict previous studies showing substantial AI productivity gains in software development.

• One prior study found AI sped up coders by 56%, while another showed developers completed 26% more tasks in a given timeframe.
• The METR study suggests these gains don’t apply universally, particularly for experienced developers working in large, established codebases they know intimately.
• Other studies often rely on AI benchmarks that may misrepresent real-world development scenarios.

Important context: The study’s scope was specifically limited to experienced developers working on familiar projects.

• The authors cautioned that results likely don’t apply to junior engineers or developers working in unfamiliar codebases.
• This distinction is crucial given predictions that AI could eliminate entry-level coding positions.
• Dario Amodei, CEO of Anthropic, recently told Axios that AI could wipe out half of all entry-level white collar jobs in the next one to five years.

Why developers still use AI: Despite the productivity slowdown, most study participants and authors continue using Cursor today.

• The tools make the development experience easier and more pleasant, comparable to “editing an essay instead of staring at a blank page.”
• “Developers have goals other than completing the task as soon as possible,” Becker explained, noting they prefer this less effortful route.

Why this matters: The findings suggest that AI’s impact on software development productivity is more nuanced than widely believed, particularly for experienced developers.

• The results could influence investment decisions in AI development tools and reshape expectations about AI’s role in replacing human engineers.
• The study highlights the importance of context-specific research when evaluating AI’s real-world effectiveness across different skill levels and project types.