← All items

Microsoft Study Finds Coding-Agent Adopters Merged 24% More Pull Requests

A study of Microsoft's own rollout of Claude Code and Copilot CLI estimates a durable output lift — and warns that merged pull requests are a proxy, not delivered value.

Stat card reading plus 24 percent more pull requests merged by coding-agent adopters, with a small peer-diffusion network motif
Chart by The Context Times. Data: the arXiv study cited below; figure not reproduced from the paper.

A paper posted July 1 studies Microsoft's early-2026 rollout of Claude Code and GitHub Copilot CLI across tens of thousands of its engineers. The authors estimate that engineers who adopted the tools merged roughly 24% more pull requests than they otherwise would have, with the lift persisting across the four-month observation window rather than fading after the novelty passed.

Two secondary findings are the more interesting ones. First use spread mainly through engineers' social networks — seeing a nearby colleague adopt an agent predicted adoption better than any top-down push — which suggests visible peer use may matter more to uptake than a mandate. Retention, meanwhile, tracked coding activity more than demographics: the engineers who kept using the tools were the ones already writing the most code, not a particular tenure or team profile.

The measurement deserves care, and the authors are explicit about it. The design is observational, not a randomized trial, so adopter and non-adopter differences can leak into the estimate. And a merged pull request is a proxy for output, not for delivered value: it says nothing about whether the change was important, whether it added review and rebase burden elsewhere, or whether it later had to be reverted. Four months is also a short window for effects like added maintenance to surface.

For enterprises now running their own rollouts, the useful question is what to instrument next. A 24% throughput number is a starting signal, not a verdict — the metrics that would confirm or puncture it are review load per change, defect and revert rates, and some measure of shipped value, tracked past the point where a productivity bump usually looks its best.

Entities

Sources