AI metrics can show:
How many and what types of developers are adopting AI tooling How much work and what kind is being touched by AI How much it costs Core engineering metrics show:
If teams are shipping faster If quality and reliability are increasing or decreasing If code maintainability is decreasing If AI tools are reducing friction in the developer workflow
I think this is great, you can measure how much engineers are utilizing AI but that doesn’t tell you how good the impact is.
This framework is utilization -> impact -> cost. You don’t want to pay for something no one is using, which is another signal as well.
Quote Citation: Gergely Orosz, “How tech companies measure the impact of AI on software development”, 2025-03-11, https://newsletter.pragmaticengineer.com/p/how-tech-companies-measure-the-impact-of-ai