Measuring the Impact of AI on Delivery Speed
The story so far
James Shore proposed that the impact of AI on software development delivery speed needs to be measured by comparing a team's performance before and after. Since deliverables are not standardized, metrics such as lines of code and PR numbers are unreliable and estimates are inaccurate. He suggests using randomized controlled trials, alternating between tasks using AI and tasks not using AI, to reduce interference from other process improvements, thereby more accurately assessing the actual impact of AI.
AI-written from the reports · Updated 14 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- O'Reilly RadarMeasuring AI’s Impact on Delivery Speed
Does AI make software development faster? Delivery speed is not productivity, but the ratio of value output to cost. Since software deliverables are not standardized, metrics like lines of code and PR numbers are unreliable, and estimations are inaccurate. Measuring delivery speed requires comparing a team's performance before and after. Through randomized controlled trials, alternating between tasks using AI and tasks not using AI can reduce interference from other process improvements, thus more accurately assessing the actual impact of AI on delivery speed.