Russell Buzby

Early this year, Microsoft rolled out the command-line AI coding agents Claude Code and GitHub Copilot CLI across its engineering workforce, and researchers tracked what happened among tens of thousands of engineers. Two findings stand out. Adopters merged roughly 24 per cent more pull requests than they otherwise would have, a lift that held across the four-month study window. And the tools spread the way gossip spreads, through peer networks rather than management decree, with first use travelling along social connections and retention predicted by how much someone actually codes (Adoption and impact of command-line AI coding agents, 2026). The study is observational rather than experimental, and its authors add their own caution: a merged pull request is not the same thing as the value it delivers. Even so, for anyone running a technology function, a durable 24 per cent lift in shipped work is the kind of number that ends debates.

A second paper, published six days later, names what the number conceals. Its authors call it Knowledge Debt, a developer-level analogue of technical debt: AI agents boost output while stripping away the incidental learning that used to come free with the work (Agents that teach, 2026). Writing code was never just production. It was how developers built mental models of the systems they own, the way walking a fireground teaches things no briefing can. Delegate the writing and the production stays; the learning goes. The paper cites an industry estimate from Sonar that roughly 42 per cent of committed code is now AI-generated or AI-assisted, projected to reach 65 per cent by 2027. Whatever precision you grant a vendor survey, the direction is not in dispute, and it means the code bases governments and companies will depend on for decades are increasingly written by something other than the people responsible for them.

The cost side has numbers of its own. A controlled study published by Anthropic in January took 52 professional developers, set them tasks in a Python library none of them knew, and compared those working with AI assistance against those working without. On the comprehension quiz that followed, the AI-assisted group scored 17 per cent lower, a two-grade-point gap on a 27-point assessment, and the steepest declines belonged to developers who delegated the coding entirely (How AI impacts skill formation, 2026). Self-perception offers no protection here. In a 2025 randomised trial by METR, sixteen experienced open-source developers using early-2025 AI tools took 19 per cent longer on real tasks in their own repositories, while believing the AI had sped them up by about 20 per cent (Becker et al., 2025). The tools have improved since, and the Microsoft data suggests today’s agents genuinely do lift throughput. The enduring lesson is the gap between felt productivity and measured effect, because the same self-assessment error applies to felt understanding. Developers seem to sense the trade in aggregate: Stack Overflow’s 2025 survey found usage climbing to 84 per cent while trust in AI output accuracy fell from 40 to 29 per cent in a year (Stack Overflow, 2025). People increasingly use tools they decreasingly trust, which is a reasonable description of a workforce accumulating a liability it can feel but cannot see.

The reason this belongs on executive agendas rather than engineering blogs is what comprehension is for. Nobody is paid to comprehend code for its own sake. Comprehension is the asset you draw on at 2 am when the system fails, when the security review asks why a dependency exists, when the regulator wants the decision logic explained, when the vendor leaves and the system stays. Those moments are exactly where AI assistance is weakest and institutional understanding is the whole game. The debt also compounds generationally. Senior engineers acquired their judgement by writing the routine code now being delegated, so a team that automates its juniors’ learning years is also automating away its future reviewers, the people qualified to check the AI. I wrote separately about the augmentation trap, where assistance erodes the expertise it depends on, and knowledge debt is that trap operating at the scale of an entire profession’s pipeline.

What makes the situation tractable is that the research points at mechanisms, and they are cheap relative to what they protect. The Knowledge Debt paper does not counsel abstinence; it proposes designing incidental learning back into the workflow through six design principles plus a prototype called SHIELD, which turns the agent’s own reasoning into brief, out-of-band learning moments without breaking the developer’s flow (Agents that teach, 2026). A companion study embedded a Socratic tutor in the IDE and measured significant gains in prompt engineering skill after one 60-minute session, albeit across only fifteen developers (Prompt Coach, 2026). A third benchmarked small language models as low-cost tutors and found the pedagogy is the hard part, with models still prone to handing over answers rather than developing the learner (CSTutorBench, 2026). The tooling exists, imperfect but improving. What it needs from leadership is the decision that learning is a designed property of the development environment, budgeted like security, rather than a by-product presumed to survive automation.

There is also a rollout lesson hiding in the Microsoft data. If adoption travels through peer networks, so does practice, good habits along with bad. An organisation that wants agents used well should invest in visible exemplars, engineers whose habits others copy, because the diffusion channel is social whatever the policy says. For public sector technology teams the stakes are compounded by ownership horizons. Government systems routinely run for twenty years, the APS is rebuilding in-house digital capability after a decade of contracting it out, and I write separately about how restructures liquidate institutional knowledge. It would be a bitter irony to rebuild the workforce while hollowing out its learning, accumulating systems the rebuilt workforce has shipped but never understood. Delivery agencies adopting coding agents, and they should adopt them, need the comprehension safeguards in the same business case as the licences.

Technical debt at least leaves a trail in the repository, where a determined team can find and refactor it. Knowledge debt accrues in people, invisible to every dashboard, discoverable only at the moment of need, which is the most expensive possible time. The 24 per cent is real, and it is bankable this quarter. The understanding it borrows against falls due later, with interest, on someone else’s watch. Ship faster by all means. But the expertise that used to arrive as a free by-product of the typing now has to be built deliberately, as a deliverable in its own right, because the alternative is a workforce that gets more productive every quarter at work it understands less every year.

References

Adoption and impact of command-line AI coding agents: A study of Microsoft’s early 2026 rollout of Claude Code and GitHub Copilot CLI. (2026, July 1). arXiv:2607.01418. https://arxiv.org/abs/2607.01418

Agents that teach: Towards designing incidental learning back into AI-assisted software development. (2026, July 7). arXiv:2607.06101. https://arxiv.org/abs/2607.06101

Becker, J., Rush, N., Barnes, E., & Rein, D. (2025). Measuring the impact of early-2025 AI on experienced open-source developer productivity. arXiv:2507.09089. https://arxiv.org/abs/2507.09089

CSTutorBench: Benchmarking small language models as tutors for block-based programming. (2026, July 6). arXiv:2607.05571. https://arxiv.org/abs/2607.05571

How AI impacts skill formation. (2026, January 28). arXiv:2601.20245. https://arxiv.org/abs/2601.20245

Prompt Coach: An empirical evaluation of an agentic tutor for learning prompt engineering in software development.(2026, July 7). arXiv:2607.06074. https://arxiv.org/abs/2607.06074

Stack Overflow. (2025). 2025 Developer Survey: AI. https://survey.stackoverflow.co/2025/ai/