10 Tools to Measure DevOps Performance Metrics
Compare 10 tools for DORA and DevOps performance metrics: Apache DevLake, GitLab, Datadog, Prometheus, Grafana, PagerDuty, LinearB and more.
The best tools for measuring DevOps performance read data you already have (Git, CI/CD, incident and ticket systems) and turn it into the DORA metrics: deployment frequency, change lead time, change fail rate and failed deployment recovery time. Open source options include Apache DevLake, Prometheus and Grafana; commercial options include Datadog, GitLab, Jellyfish, LinearB, Swarmia and Harness.
Key takeaways
- Pick the metrics first, then the tool. Most teams start with the four DORA delivery metrics and add reliability data from incidents.
- DORA renamed "mean time to recovery" to failed deployment recovery time in its 2023 report, and its 2024 report added rework rate as a fifth delivery measure.
- Open source tools (Apache DevLake, Prometheus, Grafana) cost engineering time; commercial platforms cost licence money but connect faster.
- No tool fixes bad source data. Consistent deployment events, linked commits and tagged incidents matter more than the dashboard.
- Use the numbers to find bottlenecks in the system, not to rank individual engineers.
If you are new to the metrics themselves, read the value of DORA metrics in evaluating DevOps maturity first, or start with the broader What Is DevOps guide. This post is about the tooling: what each tool measures, where its data comes from, and when it fits.
What should a DevOps performance tool measure?
DORA (DevOps Research and Assessment, now part of Google Cloud) publishes the most widely used set of software delivery metrics. They split into throughput and instability.
| Metric | Type | Question it answers | Typical data source |
|---|---|---|---|
| Deployment frequency | Throughput | How often do we ship to production? | CD tool deploy events |
| Change lead time | Throughput | How long from commit to running in production? | Git commits plus deploy events |
| Change fail rate | Instability | What share of deployments need a fix or rollback? | Deploys linked to incidents or hotfixes |
| Failed deployment recovery time | Instability | How fast do we recover from a bad deployment? | Incident open and resolve times |
| Rework rate | Instability | How many deployments were unplanned fixes? | Deploy labels or linked tickets |
Alongside these, teams usually track operational signals (latency, error rate, saturation, SLO burn) and flow signals (pull request cycle time, review wait, work in progress). For a wider metric list see top DevOps metrics for team and system performance and metrics for high-performance CI/CD workflows.
The 10 tools at a glance
| # | Tool | Licence | Best at | DORA out of the box? |
|---|---|---|---|---|
| 1 | Apache DevLake | Open source (Apache 2.0) | Self-hosted DORA dashboards from many sources | Yes |
| 2 | GitLab | Commercial tiers | DORA and value stream data inside GitLab | Yes (paid tier) |
| 3 | Datadog | Commercial SaaS | DORA next to APM and incident data | Yes |
| 4 | Prometheus | Open source (CNCF) | Collecting service and pipeline metrics | No, build it |
| 5 | Grafana | Open source plus cloud | Dashboards across many data sources | No, build it |
| 6 | PagerDuty | Commercial SaaS | Incident response and recovery time | Partial (recovery side) |
| 7 | Jellyfish | Commercial SaaS | Engineering investment and delivery reporting | Yes |
| 8 | LinearB | Commercial SaaS | Pull request flow and workflow automation | Yes |
| 9 | Swarmia | Commercial SaaS | Team-level flow and working agreements | Yes |
| 10 | Harness Software Engineering Insights | Commercial | Insights tied to a CI/CD platform | Yes |
Commercial pricing changes often and depends on seat count, so it is left out here. Check each vendor's pricing page.
1. Apache DevLake
Apache DevLake is an Apache Software Foundation project that pulls data from GitHub, GitLab, Bitbucket, Jenkins, Jira, Azure DevOps and other tools into one database, then shows it in bundled Grafana dashboards. It ships a ready DORA dashboard.
How it calculates DORA
You define which CI/CD runs count as production deployments (for example by job or environment name) and which Jira or GitHub issues count as incidents. DevLake links commits to deployments and incidents to deployments, then computes the four metrics. Most setup time goes into these rules.
When it fits
- You want DORA without sending source data to a SaaS vendor.
- You have someone who can run a database, the DevLake services and Grafana (Docker Compose or Helm).
- Your toolchain is mixed, such as GitHub code with Jenkins builds and Jira incidents.
2. GitLab DORA metrics and Value Stream Analytics
If code, pipelines and deployments all live in GitLab, it can report DORA metrics without another tool. GitLab records deployments to environments, so deployment frequency and lead time come straight from pipeline data. Change fail rate and recovery time use GitLab incidents.
Points to check
- DORA and Value Stream Analytics features sit in paid tiers. Check the tier table on docs.gitlab.com before planning around them.
- Deployments only count when jobs declare an
environment, so pipeline hygiene matters.
deploy_production:
stage: deploy
script:
- ./deploy.sh
environment:
name: production
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
For deeper GitLab practice, see GitLab CI/CD interview questions.
3. Datadog DORA Metrics
Datadog offers a DORA Metrics product that takes deployment events (from its CI Visibility or an API call from your pipeline) and failure events (from Datadog Incident Management or an API), then charts the four metrics. Its advantage is context: a spike in change fail rate sits next to APM traces, logs and monitors for the same service.
When it fits
- You already use Datadog for monitoring and want delivery metrics in the same place.
- You can tag services consistently, since Datadog groups DORA data by service and team.
4. Prometheus
Prometheus is a CNCF graduated monitoring system that scrapes metrics over HTTP and stores them as time series. It does not compute DORA metrics by itself. Teams use it in two ways:
- Operational metrics: request rate, error rate, latency and saturation, which feed SLOs and show the real impact of a deployment.
- Pipeline metrics: Jenkins (through its Prometheus plugin), Argo CD and many CI tools expose metrics you can scrape. Batch jobs can push to the Pushgateway, for example a counter incremented at the end of every production deploy.
# deployments per day over the last week, by service
sum by (service) (increase(deployments_total{env="production"}[1d]))
The metric name above is an example you would create yourself. Lead time is hard to compute in Prometheus because it needs commit timestamps, so most teams pair it with DevLake or a SaaS tool for that metric.
5. Grafana
Grafana is the dashboard layer. It reads Prometheus, Loki, SQL databases, Elasticsearch, cloud monitoring APIs and many plugin sources, so one board can show deploy frequency from a SQL table next to error rate from Prometheus. DevLake itself uses Grafana for its dashboards.
Good practice
- Add deployment annotations so every graph shows when releases happened.
- Keep one DORA overview board per team, then link out to service boards.
- Store dashboards as JSON in Git and provision them, so they are reviewed like code.
See Prometheus and Grafana together for a setup walk-through and real-time monitoring dashboards to build for board ideas.
6. PagerDuty
PagerDuty handles alert routing, on-call schedules and incident response. For performance measurement it supplies the instability side: when incidents opened, were acknowledged and were resolved, and which service and change caused them. Its analytics report on mean time to acknowledge and resolve. Feeding PagerDuty incidents into DevLake, Datadog or a commercial insights tool gives a more honest recovery time than ticket timestamps.
7. Jellyfish
Jellyfish is an engineering management platform. It reads Git, Jira and CI/CD data and reports DORA metrics along with where engineering time goes (new features, maintenance, unplanned work). It suits leaders who need to explain delivery and investment to the business, more than teams that want a pull request coach.
8. LinearB
LinearB focuses on the pull request flow: coding time, pickup time, review time and deploy time, plus DORA metrics. It also offers workflow automation in pull requests, such as labels for estimated review time or routing reviews. It suits teams whose lead time is dominated by review wait rather than pipeline speed.
9. Swarmia
Swarmia combines DORA metrics with team-level flow insights and "working agreements", which are targets a team sets for itself (for example, review pull requests within one working day) with Slack nudges when they slip. It suits teams that want to improve habits without a heavy management layer.
10. Harness Software Engineering Insights
Harness offers Software Engineering Insights (SEI) as part of its platform. It pulls from SCM, issue trackers and CI/CD (Harness or others) and reports DORA and developer flow metrics. It fits best when you already run Harness CI or CD, because deployment events are then native. See Harness CD interview questions for background on the platform.
How to choose: a decision table
| Your situation | Start with |
|---|---|
| Small budget, mixed tools, some ops capacity | Apache DevLake plus Grafana |
| Everything in GitLab, paid tier available | GitLab DORA and Value Stream Analytics |
| Already on Datadog for monitoring | Datadog DORA Metrics |
| Need SLO and runtime data, not just delivery | Prometheus plus Grafana |
| Recovery time is the weak spot | PagerDuty feeding your DORA tool |
| Leadership reporting on investment | Jellyfish |
| Slow code review drives lead time | LinearB or Swarmia |
| Already on the Harness platform | Harness SEI |
Rolling out performance measurement in five steps
- Define events. Agree what counts as a production deployment, a failure and a recovery. Write it down.
- Fix the data. Make pipelines emit a deploy event with service, version and commit SHA. Link incidents to the deploy that caused them.
- Baseline for a month. Do not set targets until you see normal variation.
- Pick one bottleneck. If lead time is long, find whether review, build or approval waits dominate, and fix that.
- Review monthly as a team. Treat metrics as a conversation starter. DORA itself warns against turning them into individual targets.
Common mistakes
- Counting every pipeline run as a deployment. Only production releases count.
- Measuring recovery from ticket close time. Tickets often close days after service was restored. Use incident resolve time.
- Comparing teams with different systems. A mobile app and an internal API will never ship at the same rate.
- Ignoring pipeline health. Flaky tests and slow builds inflate lead time. CI/CD monitoring tools cover that side.
- Buying before defining. A tool shows whatever your data says, right or wrong.
Learn the skills behind the dashboards
Measuring delivery well needs working knowledge of CI/CD, Git and monitoring. Our DevOps training covers pipelines and release practices, and the Prometheus and Grafana course covers metrics, PromQL and dashboards. Both run in our Kharadi, Pune classroom and live online. For observability tooling beyond delivery metrics, see top observability tools for DevOps monitoring.
FAQ
What are the DORA metrics?
They are deployment frequency, change lead time, change fail rate and failed deployment recovery time. DORA's 2024 report added rework rate. Together they describe how fast and how safely a team delivers software.
Is there a free tool for DORA metrics?
Yes. Apache DevLake is open source and ships a DORA dashboard. Prometheus and Grafana are also free, but you must build the DORA calculations yourself.
Can Prometheus measure deployment frequency?
Yes, if your pipeline exposes or pushes a deployment counter. It is weaker for lead time, which needs commit timestamps from Git.
What replaced MTTR in DORA?
DORA's 2023 report renamed it failed deployment recovery time, to focus on recovery from failures caused by changes rather than every outage.
Should DORA metrics be used to rate engineers?
No. They measure a team's delivery system. Using them as individual targets encourages gaming, such as splitting deploys to raise frequency.
Do I need a commercial tool?
Not to start. Many teams begin with GitLab's built-in reports or DevLake, then buy a platform when they need investment reporting or many integrations.
Which tool is best for recovery time?
An incident platform such as PagerDuty gives accurate open and resolve times. Feed it into your DORA tool rather than relying on ticket dates.
References
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0