Skip to content

Correlating a Datadog spike with a recent deploy

The engineer pivots from the Datadog monitor to the affected service's APM page and lines the spike up against the deployment markers. They capture the previous and current release tags spanning the spike and prepare the GitHub compare URL between them.

Category
Tags
datadoggithubdeployment-markersrelease-tagscorrelation
What and why
The observed behaviour and the reasoning behind it.
Behaviour
Reasoning
Cause and effect
What initiates this pattern and what it produces.
Trigger
Outcome
Standard operating procedure
Step-by-step instructions to reproduce this pattern.
1

Datadog

On the APM service page, locate the deployment markers panel below the latency graph.

The panel lists every deploy in the time range with its version tag, the duration since the previous deploy and the change in error rate attributed to that deploy. Sort by time descending so the most recent deploys are at the top.

Expected: The deployment markers panel shows a list of deploys, each labelled with its version tag and deploy time.

2

Datadog

Identify the deploy whose time falls within 5 minutes before the spike start.

Allow a 5 minute buffer in both directions. APM aggregates in 1 minute buckets, CDN and edge cache TTL adds 1 to 4 minutes of further delay and rolling deploys take 1 to 3 minutes to reach all instances. A deploy marker that lands 4 minutes before the spike is still the prime suspect.

Expected: You have a single deploy marker selected, with a version tag and a timestamp.

3

Datadog

Copy the version tag from the deploy marker.

The version tag is usually a short SHA prefix or a date-stamped semver style tag (v2026.05.06.1, 8a3f1cd, release-2026-05-06). It is the literal git tag created by the CI pipeline at deploy time, not a branch name.

Expected: The version tag is on the clipboard and visible in your scratch note.

4

GitHub

Open the affected service's repository and click 'Releases' in the right hand sidebar.

Use Releases not Pull Requests. The Pull Requests list contains everything ever merged including reverts, dependabot updates and feature flag toggles. Releases only contains what actually shipped, which is the only set that could have caused a production change.

Expected: The Releases page opens listing the most recent releases in reverse chronological order.

5

GitHub

Locate the release tag matching the version from the Datadog marker.

If the version is a SHA prefix, search the page with ⌘F. If it is a semver tag, the most recent two releases are usually the right pair regardless. Note the previous release tag immediately above your suspect tag, this is the comparison baseline.

Expected: You have identified two release tags: the suspect deploy and the release immediately before it.

6

GitHub

Construct or follow the compare URL between the two release tags.

GitHub releases have a 'Compare' link on each release that compares to the previous tag. If the link is missing, use github.com/<org>/<repo>/compare/<previous-tag>...<suspect-tag>. The triple-dot syntax is wrong here, use the double-dot to get the actual commit set in the second tag.

Expected: The GitHub compare view opens showing the merged PRs and commits between the two release tags.

Related patterns
How this pattern connects to other patterns in the library.
Supporting actions
Actions that provide evidence for this pattern.
Lined up api-gateway 5xx spike against deploy marker v2026.05.06.2
Datadog deploy marker for checkout-service matches release-2026-04-30-3
Compare URL: api-gateway/compare/v2026.05.05.1...v2026.05.06.2
Confirmed billing-worker spike correlates with deploy 8a3f1cd
payments-service: deploy marker 6 min before spike, within buffer
Metadata
Timestamps and identifiers.
EvidenceObserved 41 times across 5 connections
ApplicationsDatadog, GitHub
First seen25 Jan 2026, 15:38
Last seen5 May 2026, 17:14
Questions

Frequently asked questions

Speak to the founder

Henry Denton, founder of FusedFrames

Get a demo. Watch a live capture, then an AI agent query the result.

Ask anything. Pricing, security or integrating with your stack.

No purchase obligation

Start capturing

Record in minutes. Install once and work as normal.

Plug AI agents in. One API call from any AI agent stack.

Refund on unused credits if you cancel