DORA Metrics in Practice 2026

Why another DORA report when DORA already publishes one annually

The four DORA metrics — deployment frequency, lead time for changes, time to restore service, change failure rate — are the most-cited delivery benchmark in software engineering. The framework is rigorous; the popular interpretation often isn't. DORA explicitly does NOT claim causation between any single metric and organisational performance; the DORA 2024 report goes to considerable lengths to disclaim it. Yet the popular reading — "AI raises elite-DORA performance," "improving deployment frequency causes elite quartile membership" — routinely treats correlation as causation.

We chose to publish Volume 0 before Volume 1 for three reasons.

First, the cross-year synthesis of DORA reports rarely surfaces in industry coverage. DORA publishes annually; the year-over-year trends are visible in the side-by-side, not in any single year's report. The shift from "MTTR" to "Time to Restore Service" in the 2024 report, the addition of "Documentation" and "User-centricity" as supporting capabilities, the explicit AI cohort framing — each of these is most legible when read across years, which is what Volume 0 does.

Second, pre-registering hypotheses before fielding is the difference between a research report and a press release with footnotes. The hypotheses we register today are the hypotheses we will test when our 800-respondent segmented refresh fields. DORA's annual cadence means the most-recent comparative segment — AI cohort by team size — is not interrogated in real time; the Stride 2026 refresh fills that gap.

Third, the 2026 question DORA's annual cadence doesn't capture in real time: does AI adoption cause elite-quartile performance, or does pre-existing engineering practice maturity explain both? DORA 2024 reports correlation; the Stride 2026 Volume 1 tests whether the correlation survives controls for company size and practice maturity.

The four metrics, properly defined

The DORA framework was established by the early DORA research summarised in the Accelerate book (Forsgren, Humble & Kim 2018) and refined across seven annual reports. The current canonical definitions (from DORA 2024) are:

Deployment frequency. How often does your team deploy to production? Elite-quartile threshold: on demand (multiple deploys per day). Low-quartile threshold: less than once every six months.

Lead time for changes. How long does it take a code change to go from "committed" to "running in production"? Elite: less than one day. Low: one to six months.

Time to restore service. After a production incident is detected, how long until service is restored? (DORA renamed this metric from "Mean Time to Recovery / Repair" in 2024 to disambiguate from MTTR-as-Repair which measures engineering time on the underlying fix rather than the restoration to service.) Elite: less than one hour. Low: one week to one month.

Change failure rate (CFR). What percentage of deployments cause a degradation in service requiring remediation? Elite: 0–5%. Low: ≥15%.

Each metric has the property that improving one in isolation is achievable but not desirable. Deploying more frequently without driving down change failure rate is just shipping bugs faster. Reducing lead time without reducing CFR can hide accumulating technical debt. The framework's empirical observation: elite-quartile teams demonstrate all four metrics at the elite threshold simultaneously, and exhibit characteristic engineering practices (trunk-based development, CI/CD, test automation, on-call rotation maturity).

Seven years of DORA, side by side

DORA's annual cadence makes the framework the longest-running large-N cohort study in software-delivery research. Reading across years surfaces patterns that single-year reports flatten.

Three patterns are visible across the seven years. First, the elite-quartile share is stable at roughly 15–20% with year-over-year variance of about ±4 percentage points. The framework's reweighting in 2021 (three-tier framing) and 2024 (Time to Restore replacing Time to Recover) shifts these numbers slightly but the underlying cohort shape holds. Second, the medium quartile has compressed from a 2018 high of ~48% toward ~30% by 2024 — the population is bimodalising into "elite + high" and "low," with fewer teams sitting in the middle. Third, the low-quartile share grew from 14% (2018) to 21% (2024). DORA's commentary attributes part of this growth to the broader population now being sampled (DORA's cohort has expanded beyond the early-adopter subset).

What modern reading of the framework gets right and wrong

The framework's empirical core — four metrics, four quartiles, characteristic practice clusters at each tier — has held up well across seven years. Practitioners broadly recognise the quartile descriptors. Engineering leaders use the framework to benchmark internal teams against the published cohort.

Where the popular reading goes off the rails is on causation. The 2024 report adds AI adoption as a cross-tab variable and reports that AI usage is ~1.4× more prevalent in elite-quartile teams than low-quartile teams. The popular interpretation: AI causes elite performance. The published reading, with the explicit DORA caveat: AI adoption correlates with characteristic engineering practices already present in elite-quartile teams.

The pattern most commonly misread is "increasing your AI adoption will move you up a quartile." DORA's framework treats AI as a capability alongside practices like trunk-based development and on-call maturity, not as a standalone cause. The Stride 2026 segmented refresh tests whether the AI-quartile correlation survives controls for company size and pre-existing practice maturity.

The four landmark sources, side by side

The AI-adoption-bucket question

The single most-quoted finding from DORA 2024 — elite-quartile teams adopt AI at 1.4× the rate of low-quartile teams — admits at least three different causal stories.

Story 1: AI raises performance. Adopting AI tools (Copilot, ChatGPT, Stride) makes engineering teams measurably faster and more reliable, moving them up DORA quartiles.

Story 2: Performance enables AI. Elite-quartile teams already have the engineering practices (CI/CD, code review, test automation) that make AI adoption tractable. Low-quartile teams without those practices can't get AI to stick because the surrounding workflow can't support it.

Story 3: Selection effects. Elite-quartile teams self-select into adopting new tools faster across the board. AI adoption is a marker of "team that tries new things," not a cause of performance.

DORA 2024 explicitly does not claim Story 1. The Stride 2026 segmented refresh tests directly which story (or which combination) the data supports.

What Volume 1 will measure

The Stride 2026 DORA segmented refresh is an 800-respondent engineering-leader survey designed to test the three causal stories above. The survey uses DORA's published item wording verbatim — same quartile-classification questions — so cohort comparisons against DORA 2024 are direct.

What the design adds beyond DORA 2024:

• Tighter segmentation: industry × team size × AI-adoption-depth × regulated-vs-unregulated. DORA reports primary tables at the aggregate level; the segmented refresh runs the cross-tabs DORA's annual cadence doesn't publish.
• Practice-maturity controls: hierarchical regression with engineering-practice covariates (trunk-based development, CI/CD maturity, on-call rotation maturity, test automation maturity) to test whether the AI-quartile correlation survives controls.
• Causal-story tests: explicit operationalisation of the three stories above as testable hypotheses (H1–H3 below).

Pre-registered hypotheses

These are the five hypotheses we register on the Open Science Framework before fielding closes. They are the hypotheses we test in Q4 2026.

1. H1 — Quartile correlation weakens with controls. Elite-quartile DORA performance will correlate with high AI adoption (replicating DORA 2024 directionally), but correlation strength will weaken by ≥30% after controlling for company size and pre-existing engineering-practice maturity. Test: hierarchical linear regression; Cohen's f² effect sizes; Wilson 95% CIs.
2. H2 — Heavy AI raises CFR in non-elite quartiles. Change failure rate will be statistically higher in heavy-AI cohorts than selective-AI cohorts within the lower three quartiles. The elite quartile will not show this gap. Test: ANOVA across cohorts × quartile.
3. H3 — Deployment frequency improvements concentrate in small teams. Year-over-year deployment-frequency gains will be larger for teams ≤50 (≥30% improvement) than teams 5,000+ (≤10% improvement). Test: difference-in-differences regression.
4. H4 (null) — Time to Restore is practice-driven, not AI-driven. Time to Restore service will not differ between AI-adopted and AI-opted-out cohorts after controlling for team practice maturity (CI/CD, on-call rotation, runbook quality). The null is the finding. Test: ANCOVA with practice covariates.
5. H5 (exploratory) — Two-factor explanation. A two-factor model (engineering practices × AI-adoption depth) will explain ≥80% of DORA quartile membership variance, with practices alone explaining ≥60%. Exploratory framing.

Methodology summary

The full methodology is on the companion page. The short version:

• Survey instrument: ~52 substantive items + DORA's published quartile questions verbatim. Median completion 13 minutes.
• Recruitment: 800 engineering leaders via Prolific Academic + organic top-up. Balanced across team size (1-9, 10-49, 50-499, 500-4999, 5000+), industry, regulated/unregulated, AI-adoption depth.
• Statistics: Wilson 95% CIs; Cohen's h for proportions; Cohen's d + f² for continuous. Benjamini–Hochberg FDR correction for the planned hypothesis family.
• Practice-maturity scoring: 12-item composite covering trunk-based development, CI/CD, code review, test automation, on-call rotation, post-incident review, runbook quality.
• Pre-registration: will be cross-registered on OSF before fielding closes; the link will appear here once registration completes.

Dataset publication (Volume 1)

When Volume 1 lands, the survey dataset publishes under CC-BY-4.0: anonymised individual-response CSV with weighted-indicator column, pre-computed cross-tab tables (industry × quartile, team-size × quartile, AI-depth × quartile, regulated × quartile, practice-maturity × quartile), JSON-Schema data dictionary. Single ZIP bundle + Zenodo DOI for permanent citability.

Limitations and what to expect

• Self-reported quartile classification. Like DORA itself, the Stride refresh asks respondents to self-classify against the four metrics. Self-classification is known to skew slightly toward higher quartiles; a sensitivity check against telemetry-derived classification will be reported.
• English-language, predominantly Western sample. Like the State-of-AI Volume 0, the panel arm skews US/UK/EU.
• Cross-sectional only. The Stride refresh measures a single time point. Causal claims require longitudinal data; Volume 1 will not claim causation any more than DORA does, even where the segmentation suggests it directionally.
• The literature is moving. A new DORA report typically publishes annually around October; if DORA 2025 lands between Volume 0 and Volume 1, we will append a "Recent developments" note rather than rewriting the body.

Participate in Volume 1

If you are an engineering leader and would like to participate in the Volume 1 segmented refresh (Prolific arm or organic arm), reach out to research@newlightai.com.

References

1. Forsgren, N., Humble, J. & Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution.
2. DORA Accelerate State of DevOps Report 2024. Google DORA.
3. DORA Accelerate State of DevOps Reports 2018–2023. Google Cloud + DORA.
4. Winters, T., Manshreck, T. & Wright, H. (2020). Software Engineering at Google. O'Reilly.
5. Humble, J. & Farley, D. (2010). Continuous Delivery. Addison-Wesley.
6. Kim, G., Behr, K. & Spafford, G. (2013). The Phoenix Project. IT Revolution.
7. Stack Overflow Developer Survey 2024. Stack Overflow.

Frequently asked questions

Related work

Reference this Volume 0 in your own writing using the citation below.

Stride Research. (2026). DORA Metrics in Practice 2026 — Volume 0: Seven-year synthesis and pre-registered segmented refresh. Newlight Solutions. https://www.stride.page/research/dora-metrics-in-practice-2026