Dora Metrics And Devops Performance

DORA Metrics and DevOps Performance

Purpose

DORA (DevOps Research and Assessment) metrics are the industry standard for measuring software delivery performance. Google's research across thousands of organizations identified four key metrics that predict:

• Organizational performance (profitability, market share)
• Non-commercial performance (quality, customer satisfaction)
• Team well-being and reduced burnout

Elite performers who meet reliability targets are 2.3x more likely to use trunk-based development and continuous delivery practices.

Features

The Four Key Metrics

1. Deployment Frequency

Definition: How often your organization deploys code to production.

// Deployment frequency calculation
interface DeploymentData {
  timestamp: Date;
  environment: string;
  service: string;
  success: boolean;
}

function calculateDeploymentFrequency(
  deployments: DeploymentData[],
  periodDays: number = 30
): { frequency: string; deploymentsPerDay: number } {
  const productionDeployments = deployments.filter(
    d => d.environment === 'production' && d.success
  );

  const deploymentsPerDay = productionDeployments.length / periodDays;

  let frequency: string;
  if (deploymentsPerDay >= 1) {
    frequency = 'elite'; // Multiple times per day or daily
  } else if (deploymentsPerDay >= 1/7) {
    frequency = 'high'; // Weekly to daily
  } else if (deploymentsPerDay >= 1/30) {
    frequency = 'medium'; // Monthly to weekly
  } else {
    frequency = 'low'; // Less than monthly
  }

  return { frequency, deploymentsPerDay };
}

2. Lead Time for Changes

Definition: Time from code commit to code running in production.

// Lead time calculation
interface ChangeData {
  commitTimestamp: Date;
  deployTimestamp: Date;
  commitSha: string;
  prNumber?: number;
}

function calculateLeadTime(changes: ChangeData[]): {
  medianHours: number;
  p90Hours: number;
  performance: string;
} {
  const leadTimes = changes.map(c =>
    (c.deployTimestamp.getTime() - c.commitTimestamp.getTime()) / (1000 * 60 * 60)
  );

  leadTimes.sort((a, b) => a - b);

  const median = leadTimes[Math.floor(leadTimes.length / 2)];
  const p90 = leadTimes[Math.floor(leadTimes.length * 0.9)];

  let performance: string;
  if (median < 1) {
    performance = 'elite'; // Less than 1 hour
  } else if (median < 24) {
    performance = 'high'; // Less than 1 day
  } else if (median < 168) {
    performance = 'medium'; // Less than 1 week
  } else {
    performance = 'low'; // More than 1 week
  }

  return { medianHours: median, p90Hours: p90, performance };
}

3. Change Failure Rate

Definition: Percentage of deployments that result in degraded service requiring remediation.

// Change failure rate calculation
interface DeploymentOutcome {
  deploymentId: string;
  timestamp: Date;
  success: boolean;
  causedIncident: boolean;
  requiredRollback: boolean;
  requiredHotfix: boolean;
}

function calculateChangeFailureRate(deployments: DeploymentOutcome[]): {
  rate: number;
  performance: string;
} {
  const total = deployments.length;
  const failures = deployments.filter(d =>
    d.causedIncident || d.requiredRollback || d.requiredHotfix
  ).length;

  const rate = (failures / total) * 100;

  let performance: string;
  if (rate <= 15) {
    performance = 'elite'; // 0-15%
  } else if (rate <= 30) {
    performance = 'high'; // 16-30%
  } else if (rate <= 45) {
    performance = 'medium'; // 31-45%
  } else {
    performance = 'low'; // 46%+
  }

  return { rate, performance };
}

4. Time to Restore Service (MTTR)

Definition: How long it takes to restore service when an incident occurs.

// MTTR calculation
interface Incident {
  id: string;
  startTime: Date;
  resolvedTime: Date;
  severity: 'critical' | 'major' | 'minor';
  service: string;
}

function calculateMTTR(incidents: Incident[]): {
  medianHours: number;
  performance: string;
  byService: Record<string, number>;
} {
  const restorationTimes = incidents.map(i =>
    (i.resolvedTime.getTime() - i.startTime.getTime()) / (1000 * 60 * 60)
  );

  restorationTimes.sort((a, b) => a - b);
  const median = restorationTimes[Math.floor(restorationTimes.length / 2)];

  let performance: string;
  if (median < 1) {
    performance = 'elite'; // Less than 1 hour
  } else if (median < 24) {
    performance = 'high'; // Less than 1 day
  } else if (median < 168) {
    performance = 'medium'; // Less than 1 week
  } else {
    performance = 'low'; // More than 1 week
  }

  // Group by service
  const byService: Record<string, number[]> = {};
  for (const incident of incidents) {
    if (!byService[incident.service]) byService[incident.service] = [];
    const hours = (incident.resolvedTime.getTime() - incident.startTime.getTime()) / (1000 * 60 * 60);
    byService[incident.service].push(hours);
  }

  const serviceMedians: Record<string, number> = {};
  for (const [service, times] of Object.entries(byService)) {
    times.sort((a, b) => a - b);
    serviceMedians[service] = times[Math.floor(times.length / 2)];
  }

  return { medianHours: median, performance, byService: serviceMedians };
}

Performance Levels (2024 Benchmarks)

Key Insight (2024 DORA Report): Elite performers are 2.3x more likely to meet reliability targets when using trunk-based development.

Measurement Implementation

GitHub Actions DORA Workflow

# .github/workflows/dora-metrics.yml
name: DORA Metrics Collection

on:
  schedule:
    - cron: '0 0 * * 0'  # Weekly on Sunday
  workflow_dispatch:

jobs:
  collect-metrics:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Collect Deployment Data
        id: deployments
        uses: actions/github-script@v7
        with:
          script: |
            const thirtyDaysAgo = new Date();
            thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30);

            // Get workflow runs (deployments)
            const { data: runs } = await github.rest.actions.listWorkflowRuns({
              owner: context.repo.owner,
              repo: context.repo.repo,
              workflow_id: 'deploy.yml',
              created: `>=${thirtyDaysAgo.toISOString()}`,
              status: 'completed'
            });

            const deployments = runs.workflow_runs.filter(r =>
              r.conclusion === 'success'
            );

            // Calculate deployment frequency
            const deploymentsPerDay = deployments.length / 30;

            return {
              count: deployments.length,
              perDay: deploymentsPerDay.toFixed(2),
              frequency: deploymentsPerDay >= 1 ? 'elite' :
                         deploymentsPerDay >= 0.14 ? 'high' :
                         deploymentsPerDay >= 0.03 ? 'medium' : 'low'
            };

      - name: Collect Lead Time Data
        id: lead-time
        uses: actions/github-script@v7
        with:
          script: |
            const thirtyDaysAgo = new Date();
            thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30);

            // Get merged PRs
            const { data: prs } = await github.rest.pulls.list({
              owner: context.repo.owner,
              repo: context.repo.repo,
              state: 'closed',
              sort: 'updated',
              direction: 'desc',
              per_page: 100
            });

            const mergedPRs = prs.filter(pr =>
              pr.merged_at &&
              new Date(pr.merged_at) > thirtyDaysAgo
            );

            const leadTimes = mergedPRs.map(pr => {
              const created = new Date(pr.created_at);
              const merged = new Date(pr.merged_at);
              return (merged - created) / (1000 * 60 * 60); // hours
            });

            leadTimes.sort((a, b) => a - b);
            const median = leadTimes[Math.floor(leadTimes.length / 2)] || 0;

            return {
              medianHours: median.toFixed(1),
              performance: median < 1 ? 'elite' :
                          median < 24 ? 'high' :
                          median < 168 ? 'medium' : 'low'
            };

      - name: Generate Report
        run: |
          cat << EOF > dora-report.md
          # DORA Metrics Report
          **Period:** Last 30 days
          **Generated:** $(date -u +"%Y-%m-%d %H:%M:%S UTC")

          ## Metrics Summary

          | Metric | Value | Performance |
          |--------|-------|-------------|
          | Deployment Frequency | ${{ fromJson(steps.deployments.outputs.result).perDay }}/day | ${{ fromJson(steps.deployments.outputs.result).frequency }} |
          | Lead Time for Changes | ${{ fromJson(steps.lead-time.outputs.result).medianHours }} hours | ${{ fromJson(steps.lead-time.outputs.result).performance }} |

          ## Recommendations
          $(if [ "${{ fromJson(steps.deployments.outputs.result).frequency }}" != "elite" ]; then echo "- Increase deployment frequency through smaller, more frequent releases"; fi)
          $(if [ "${{ fromJson(steps.lead-time.outputs.result).performance }}" != "elite" ]; then echo "- Reduce lead time by automating more of the review process"; fi)
          EOF

      - name: Upload Report
        uses: actions/upload-artifact@v4
        with:
          name: dora-metrics-report
          path: dora-report.md

Custom Metrics Collection Script

// scripts/collect-dora-metrics.ts
import { Octokit } from '@octokit/rest';

interface DORAMetrics {
  period: { start: Date; end: Date };
  deploymentFrequency: {
    count: number;
    perDay: number;
    performance: 'elite' | 'high' | 'medium' | 'low';
  };
  leadTime: {
    medianHours: number;
    p90Hours: number;
    performance: 'elite' | 'high' | 'medium' | 'low';
  };
  changeFailureRate: {
    total: number;
    failures: number;
    rate: number;
    performance: 'elite' | 'high' | 'medium' | 'low';
  };
  mttr: {
    medianHours: number;
    incidentCount: number;
    performance: 'elite' | 'high' | 'medium' | 'low';
  };
  overallPerformance: 'elite' | 'high' | 'medium' | 'low';
}

class DORAMetricsCollector {
  private octokit: Octokit;
  private owner: string;
  private repo: string;

  constructor(token: string, owner: string, repo: string) {
    this.octokit = new Octokit({ auth: token });
    this.owner = owner;
    this.repo = repo;
  }

  async collect(periodDays: number = 30): Promise<DORAMetrics> {
    const end = new Date();
    const start = new Date();
    start.setDate(start.getDate() - periodDays);

    const [deployments, prs, incidents] = await Promise.all([
      this.getDeployments(start, end),
      this.getMergedPRs(start, end),
      this.getIncidents(start, end)
    ]);

    // Calculate each metric
    const deploymentFrequency = this.calcDeploymentFrequency(deployments, periodDays);
    const leadTime = this.calcLeadTime(prs);
    const changeFailureRate = this.calcChangeFailureRate(deployments, incidents);
    const mttr = this.calcMTTR(incidents);

    // Determine overall performance
    const performances = [
      deploymentFrequency.performance,
      leadTime.performance,
      changeFailureRate.performance,
      mttr.performance
    ];

    const overallPerformance = this.getOverallPerformance(performances);

    return {
      period: { start, end },
      deploymentFrequency,
      leadTime,
      changeFailureRate,
      mttr,
      overallPerformance
    };
  }

  private async getDeployments(start: Date, end: Date) {
    const { data } = await this.octokit.actions.listWorkflowRuns({
      owner: this.owner,
      repo: this.repo,
      workflow_id: 'deploy.yml',
      created: `${start.toISOString()}..${end.toISOString()}`
    });
    return data.workflow_runs;
  }

  private async getMergedPRs(start: Date, end: Date) {
    const { data } = await this.octokit.pulls.list({
      owner: this.owner,
      repo: this.repo,
      state: 'closed',
      sort: 'updated',
      per_page: 100
    });
    return data.filter(pr =>
      pr.merged_at &&
      new Date(pr.merged_at) >= start &&
      new Date(pr.merged_at) <= end
    );
  }

  private async getIncidents(start: Date, end: Date) {
    // This would typically come from PagerDuty, OpsGenie, or GitHub Issues
    // Placeholder implementation
    const { data } = await this.octokit.issues.listForRepo({
      owner: this.owner,
      repo: this.repo,
      labels: 'incident',
      state: 'closed',
      since: start.toISOString()
    });
    return data;
  }

  private calcDeploymentFrequency(deployments: any[], periodDays: number) {
    const successful = deployments.filter(d => d.conclusion === 'success');
    const perDay = successful.length / periodDays;

    return {
      count: successful.length,
      perDay,
      performance: this.getFrequencyPerformance(perDay)
    };
  }

  private calcLeadTime(prs: any[]) {
    const times = prs.map(pr => {
      const created = new Date(pr.created_at);
      const merged = new Date(pr.merged_at);
      return (merged.getTime() - created.getTime()) / (1000 * 60 * 60);
    });

    times.sort((a, b) => a - b);
    const median = times[Math.floor(times.length / 2)] || 0;
    const p90 = times[Math.floor(times.length * 0.9)] || 0;

    return {
      medianHours: median,
      p90Hours: p90,
      performance: this.getLeadTimePerformance(median)
    };
  }

  private calcChangeFailureRate(deployments: any[], incidents: any[]) {
    const total = deployments.filter(d => d.conclusion === 'success').length;
    const failures = incidents.length; // Simplified

    const rate = total > 0 ? (failures / total) * 100 : 0;

    return {
      total,
      failures,
      rate,
      performance: this.getFailureRatePerformance(rate)
    };
  }

  private calcMTTR(incidents: any[]) {
    const times = incidents
      .filter(i => i.closed_at)
      .map(i => {
        const opened = new Date(i.created_at);
        const closed = new Date(i.closed_at);
        return (closed.getTime() - opened.getTime()) / (1000 * 60 * 60);
      });

    times.sort((a, b) => a - b);
    const median = times[Math.floor(times.length / 2)] || 0;

    return {
      medianHours: median,
      incidentCount: incidents.length,
      performance: this.getMTTRPerformance(median)
    };
  }

  private getFrequencyPerformance(perDay: number): 'elite' | 'high' | 'medium' | 'low' {
    if (perDay >= 1) return 'elite';
    if (perDay >= 1/7) return 'high';
    if (perDay >= 1/30) return 'medium';
    return 'low';
  }

  private getLeadTimePerformance(hours: number): 'elite' | 'high' | 'medium' | 'low' {
    if (hours < 1) return 'elite';
    if (hours < 24) return 'high';
    if (hours < 168) return 'medium';
    return 'low';
  }

  private getFailureRatePerformance(rate: number): 'elite' | 'high' | 'medium' | 'low' {
    if (rate <= 15) return 'elite';
    if (rate <= 30) return 'high';
    if (rate <= 45) return 'medium';
    return 'low';
  }

  private getMTTRPerformance(hours: number): 'elite' | 'high' | 'medium' | 'low' {
    if (hours < 1) return 'elite';
    if (hours < 24) return 'high';
    if (hours < 168) return 'medium';
    return 'low';
  }

  private getOverallPerformance(performances: string[]): 'elite' | 'high' | 'medium' | 'low' {
    const scores = { elite: 4, high: 3, medium: 2, low: 1 };
    const avg = performances.reduce((sum, p) => sum + scores[p as keyof typeof scores], 0) / performances.length;

    if (avg >= 3.5) return 'elite';
    if (avg >= 2.5) return 'high';
    if (avg >= 1.5) return 'medium';
    return 'low';
  }
}

// Usage
const collector = new DORAMetricsCollector(
  process.env.GITHUB_TOKEN!,
  'myorg',
  'myrepo'
);

const metrics = await collector.collect(30);
console.log(JSON.stringify(metrics, null, 2));

Grafana Dashboard Configuration

{
  "dashboard": {
    "title": "DORA Metrics Dashboard",
    "panels": [
      {
        "title": "Deployment Frequency",
        "type": "stat",
        "targets": [
          {
            "expr": "sum(increase(deployments_total{environment=\"production\"}[30d])) / 30",
            "legendFormat": "Deploys/day"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "thresholds": {
              "steps": [
                { "value": 0, "color": "red" },
                { "value": 0.03, "color": "orange" },
                { "value": 0.14, "color": "yellow" },
                { "value": 1, "color": "green" }
              ]
            }
          }
        }
      },
      {
        "title": "Lead Time for Changes",
        "type": "stat",
        "targets": [
          {
            "expr": "histogram_quantile(0.5, sum(rate(lead_time_hours_bucket[30d])) by (le))",
            "legendFormat": "Median (hours)"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "unit": "h",
            "thresholds": {
              "steps": [
                { "value": 0, "color": "green" },
                { "value": 1, "color": "yellow" },
                { "value": 24, "color": "orange" },
                { "value": 168, "color": "red" }
              ]
            }
          }
        }
      },
      {
        "title": "Change Failure Rate",
        "type": "gauge",
        "targets": [
          {
            "expr": "sum(deployments_failed_total) / sum(deployments_total) * 100",
            "legendFormat": "Failure Rate %"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "unit": "percent",
            "min": 0,
            "max": 100,
            "thresholds": {
              "steps": [
                { "value": 0, "color": "green" },
                { "value": 15, "color": "yellow" },
                { "value": 30, "color": "orange" },
                { "value": 45, "color": "red" }
              ]
            }
          }
        }
      },
      {
        "title": "Time to Restore (MTTR)",
        "type": "stat",
        "targets": [
          {
            "expr": "histogram_quantile(0.5, sum(rate(incident_resolution_hours_bucket[30d])) by (le))",
            "legendFormat": "Median (hours)"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "unit": "h",
            "thresholds": {
              "steps": [
                { "value": 0, "color": "green" },
                { "value": 1, "color": "yellow" },
                { "value": 24, "color": "orange" },
                { "value": 168, "color": "red" }
              ]
            }
          }
        }
      }
    ]
  }
}

Tools and Platforms

Four Keys Setup (Google)

# Deploy Four Keys to GCP
git clone https://github.com/dora-team/fourkeys.git
cd fourkeys

# Configure
export PROJECT_ID="my-project"
export REGION="us-central1"

# Deploy
./setup/setup.sh

# Configure webhook for GitHub events
# Add to GitHub repo settings: https://<REGION>-<PROJECT_ID>.cloudfunctions.net/github-parser

Improvement Strategies

Improving Deployment Frequency

Improving Lead Time

Reducing Change Failure Rate

Reducing MTTR

Best Practices

1. Measure Consistently

// Standardized metric definitions
const METRIC_DEFINITIONS = {
  deploymentFrequency: {
    source: 'GitHub Actions',
    filter: 'workflow=deploy.yml, conclusion=success',
    aggregation: 'count per day'
  },
  leadTime: {
    source: 'GitHub PRs',
    measurement: 'created_at to merged_at',
    aggregation: 'median'
  },
  changeFailureRate: {
    source: 'GitHub Issues + Deployments',
    filter: 'label=incident, within 24h of deployment',
    aggregation: 'incidents / deployments * 100'
  },
  mttr: {
    source: 'PagerDuty',
    measurement: 'triggered_at to resolved_at',
    aggregation: 'median'
  }
};

2. Set Realistic Goals

# Quarterly improvement targets
q1_2024:
  deployment_frequency:
    current: 0.5/day
    target: 1.0/day
    improvement: 100%
  lead_time:
    current: 48h
    target: 24h
    improvement: 50%
  change_failure_rate:
    current: 25%
    target: 20%
    improvement: 20%
  mttr:
    current: 4h
    target: 2h
    improvement: 50%

3. Avoid Gaming Metrics

Use Cases

1. Team Performance Review

// Quarterly DORA review
async function quarterlyReview(team: string) {
  const metrics = await collectMetrics({ team, period: '90d' });

  return {
    summary: {
      overallPerformance: metrics.overallPerformance,
      strongestMetric: findStrongest(metrics),
      improvementArea: findWeakest(metrics)
    },
    comparison: {
      vsLastQuarter: await compareToLastQuarter(team, metrics),
      vsIndustry: compareToIndustryBenchmarks(metrics)
    },
    recommendations: generateRecommendations(metrics)
  };
}

2. DevOps Transformation Tracking

// Track transformation progress
const transformationGoals = {
  phase1: { // Foundation
    deploymentFrequency: 'weekly',
    leadTime: '< 1 week'
  },
  phase2: { // Acceleration
    deploymentFrequency: 'daily',
    leadTime: '< 1 day',
    changeFailureRate: '< 30%'
  },
  phase3: { // Excellence
    deploymentFrequency: 'multiple/day',
    leadTime: '< 1 hour',
    changeFailureRate: '< 15%',
    mttr: '< 1 hour'
  }
};

Related Skills

• devops/github-actions - CI/CD automation
• devops/observability - Monitoring and metrics
• testing/comprehensive-testing - Quality gates
• devops/feature-flags - Progressive delivery

Think Omega. Build Omega. Be Omega.

Converted and distributed by TomeVault — claim your Tome and manage your conversions.