For software engineering in particular, Peng et al. found that crowd-sourced software developers using copilot completed a task 55.5% faster than the control group and novice programmers benefited more from AI coding assistance. Follow-up studies of developers in major software companies and found that AI-generated code completions provide a 26. 8% boost in productivity as measured by pull requests, commits, and software product builds [Cui et al., 2024]. This study also found that less experienced coders experienced greater boosts in productivity.
This is the study that has been making the rounds with teh ‘gotcha’ being people who use AI don’t remember what they wrote. I don’t remember what I had for lunch yesterday…
Quote Citation: Judy Hanwen Shen Alex Tamkin, “https://arxiv.org/pdf/2601.20245", February 3, 2026, https://arxiv.org/pdf/2601.20245