PagerDuty and CodeWords: fewer pages, better ones

Every organisation that has used PagerDuty for more than a year has the same problem, and it is not a technical one. People are being woken up for things that did not need them, and the number of pages has become a thing to survive rather than a signal to respond to.

The automation that helps is not more integration. It is reducing what pages, enriching what remains so the responder starts informed, and reporting honestly on what the rotation is actually costing.

What we'll cover

• Deduplication keys, which do most of the work
• What PagerDuty reaches
• Connecting it to CodeWords
• Seven automations worth building
• Enrichment for three in the morning
• Reporting on the rotation honestly
• What should never be automated
• Schedules, and the gaps in them
• Building it so it survives
• Limits worth knowing about
• What to build first
• Frequently asked questions

Deduplication keys, which do most of the work

The Events API takes a deduplication key, and using it well is the single largest lever on page volume.

The same key updates an existing incident rather than opening a new one, so a flapping condition produces one incident instead of forty.

Choose the key to represent the condition, not the occurrence. The service plus the check plus the affected component is usually right; anything including a timestamp is always wrong.

Resolve with the same key when the condition clears, which is what makes auto-resolution work and what stops stale incidents accumulating.

Too broad a key merges unrelated problems into one incident, which is its own failure — two different outages appearing as one.

Most alert storms are a deduplication key problem, not a monitoring problem. It is worth checking before adding suppression rules on top.

What PagerDuty reaches

The Events API triggers, acknowledges, and resolves incidents, with deduplication keys, severity, and custom details.

Incidents can be read, updated, reassigned, merged, and annotated through the REST API.

Services and integrations can be configured, which suits provisioning a new service consistently.

Schedules and on-call can be queried — who is on call now, and who will be — which is what most cross-system automation actually needs.

Escalation policies can be read and managed.

Notes can be added to incidents, which is where automated context belongs.

Analytics endpoints report incident volume, response times, and patterns, which is the basis of honest rotation reporting.

Webhooks fire on incident lifecycle events.

Connecting it to CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which incidents, what should be gathered, and where it should go.
3. Authorize the connection with an API key scoped to what the workflow needs, plus a routing key for the specific service where you send events.
4. Describe the exceptions: an incident already resolved, an enrichment source that is itself down, a deduplication key that matches nothing.
5. Test against a non-production service before anything reaches the on-call rotation.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Seven automations worth building

Incident enrichment. Gather what the responder would gather anyway and attach it as a note within seconds of the page. Covered below, and it is the one with the clearest return.

Business hours routing. Something that genuinely needs attention now pages; something that can wait creates a ticket for the morning. The distinction is a business decision and it belongs in automation rather than in people's heads.

Tell the people who need to know. Support and account teams learn about incidents from customers otherwise, which is an avoidable and repeated embarrassment.

Incident record creation. A document or ticket created on major incidents with the timeline started, so the review afterwards has something to work from.

On-call visibility elsewhere. Publish who is on call into chat and into your internal directory, so nobody has to open PagerDuty to find out.

Rotation reporting. Pages per person, out-of-hours pages, and how many resulted in action. Uncomfortable and necessary.

Noisy service reporting. Which services page most, and which pages resolve themselves without anybody doing anything, which is the list of things to fix or stop alerting on.

Enrichment for three in the morning

A page at three in the morning is answered by somebody with no context and reduced judgement. Everything you can gather in advance is worth gathering.

What changed recently. Deployments, configuration changes, and feature flag changes in the last few hours, from the systems that hold them. This resolves a large share of incidents on its own.

Whether it has happened before. Previous incidents with the same deduplication key and how they were resolved. The single most useful item and the one nobody has time to look up at that hour.

The current state of related systems. Dependencies, related alerts, and whether the provider has an active status page entry.

Who is affected. Named customers or affected volume, which changes how the incident is handled and who else needs telling.

The runbook link, if one exists, and an honest note if one does not.

Attach it as a note within seconds of the incident opening. The responder reads one thing rather than opening six tabs, and the difference in time to diagnosis is substantial.

Reporting on the rotation honestly

Count out-of-hours pages per person. This is the number that predicts whether somebody leaves, and it is rarely tracked.

Report how many pages led to action. A page that resolved itself with no intervention was an interruption, not an alert.

Identify the top noisy services, since the distribution is always heavily skewed and a small number of services generate most of the pain.

Track time to acknowledge separately from time to resolve. The first measures whether the rotation is working; the second measures the problem.

Report per team and per service, not per individual, except for the out-of-hours count, which is about people rather than process.

Show it to whoever can decide. Rotation load is a management problem, and the data usually makes a case that months of complaining has not.

What should never be automated

Automatic acknowledgement. Acknowledgement means a person is handling it. Automating it removes the only signal escalation depends on.

Automatic resolution of anything you did not verify. Resolving because a metric recovered is reasonable when the metric genuinely represents the condition, and dangerous when it does not.

Reassigning away from whoever is on call, unless a person decided that.

Changing escalation policies automatically, which is configuration and belongs in a reviewed change.

Suppressing pages to make numbers look better. If a page is not worth waking somebody for, stop sending it, rather than sending it and hiding it.

Schedules, and the gaps in them

On-call schedules drift out of correspondence with reality, and the failures are quiet until they are not.

Check for gaps. A rotation with nobody assigned for a period is a page that escalates to whoever is at the end of the policy, at whatever hour it happens.

Check the escalation policy ends somewhere. A policy whose final step is a schedule with nobody on it terminates in silence.

Check overrides have been removed. Somebody covering a holiday who never had the override removed keeps receiving pages months later.

Check people who have left. Departed employees remain on schedules for surprisingly long, and the page goes nowhere.

Publish next week's rotation to the team, since the most common cause of a missed page is somebody not knowing they were on call.

A weekly check of these five is quick to build, reads only, and catches the failure where the alerting worked perfectly and nobody was told.

Building it so it survives

Choose deduplication keys deliberately, since this decision determines page volume more than anything else you do.

Make enrichment fail gracefully. An enrichment source being down should produce a note saying so, not a failed or delayed page.

Do not page about your own automation. It should degrade quietly and report through a channel that is not the on-call rotation.

Handle at-least-once webhooks, since incident events repeat.

Report the outcome. Incidents enriched, sources that failed, and anything skipped.

Limits worth knowing about

Events API and REST API are separate with different authentication — a routing key per service for events, an API key for the REST API — which is the most common initial confusion.

Rate limits apply to both, and bulk historical analysis needs pacing.

Analytics data is aggregated and may lag, so reporting is not real-time.

Some capabilities are plan-dependent, including parts of the analytics and event orchestration features.

Incident merging is one-way. Merged incidents cannot be unmerged, so automation that merges should be conservative.

What to build first

Incident enrichment: when an incident opens, gather recent deployments, previous incidents with the same key, and affected customers, and attach them as a note. It changes nothing about who is paged, it takes little building, and it measurably shortens the time between somebody waking up and somebody understanding.

Two habits make the difference. Attach it as a note rather than trying to modify the page, so it works with any notification path. And include previous occurrences first, since "this happened three weeks ago and restarting the consumer fixed it" is the single most valuable sentence a responder can read.

Frequently asked questions

How do I reduce alert noise?

Start with deduplication keys, which cause most alert storms. Then remove pages that consistently resolve without intervention. Suppression rules on top of bad keys treat the symptom and leave the cause.

Should automation acknowledge incidents?

No. Acknowledgement means a person is on it, and escalation depends on that signal. Automating it means an unattended incident looks attended, which is the worst possible failure for an on-call system.

Can automation resolve incidents?

Where the condition is genuinely verified as cleared, yes. Resolving because a metric recovered is fine when the metric represents the condition accurately, and risky otherwise. Auto-resolution through the same deduplication key is the clean version of this.

What should an enrichment note contain?

Previous occurrences and their resolutions first, then recent deployments, then affected customers, then the runbook link. That order matches what a responder needs at three in the morning.

Why are there two different credentials?

Because the Events API and the REST API are separate: a routing key identifies the service you send events to, and an API key authenticates management calls. It is the most common first stumble and it is entirely a naming problem.

What rotation metric matters most?

Out-of-hours pages per person, and how many led to action. The first predicts burnout, the second tells you how much of it was necessary. Both are easy to compute and rarely reported.

Should support know about incidents automatically?

Yes. Support and account teams otherwise find out from customers, which is avoidable and happens repeatedly. A summary in their channel at incident open, updated at resolution, is a short automation with a disproportionate effect.

How do I check the on-call schedule is actually sound?

Look for gaps with nobody assigned, escalation policies ending at an empty schedule, stale overrides, and departed employees still in rotations. All four are readable through the API and all four produce a page that reaches nobody.

Why did an incident escalate all the way with no response?

Usually a schedule gap or a stale override — the alerting worked and the notification went to somebody who was not there. It is worth checking weekly, because nothing surfaces it until an incident does.

Related reading

• Datadog and CodeWords
• Sentry webhooks
• CodeWords integrations and templates