Managing Pagerduty Events

PagerDuty Events API v2 Skill

Send and manage events using the PagerDuty Events API v2.

API Conventions

Authentication

Events API v2 uses integration/routing keys in the payload — not Authorization headers. Keys are injected by connection.

Base URL

• Events API: https://events.pagerduty.com/v2/
• Change Events: https://events.pagerduty.com/v2/change/enqueue
• This is DIFFERENT from the PagerDuty REST API (api.pagerduty.com).

Output Rules

• TOKEN EFFICIENCY: Target <=50 lines per output
• Events API returns minimal responses — extract dedup_key and status
• NEVER expose routing keys in output

Core Helper Function

#!/bin/bash

pd_event() {
    local action="$1"      # trigger, acknowledge, resolve
    local dedup_key="$2"
    local summary="${3:-}"
    local severity="${4:-error}"  # critical, error, warning, info
    local source="${5:-cloudskills}"

    local payload=""
    if [ "$action" = "trigger" ]; then
        payload="{
            \"routing_key\": \"${PD_ROUTING_KEY}\",
            \"event_action\": \"trigger\",
            \"dedup_key\": \"${dedup_key}\",
            \"payload\": {
                \"summary\": \"${summary}\",
                \"severity\": \"${severity}\",
                \"source\": \"${source}\",
                \"timestamp\": \"$(date -u +%Y-%m-%dT%H:%M:%S.000Z)\"
            }
        }"
    else
        payload="{
            \"routing_key\": \"${PD_ROUTING_KEY}\",
            \"event_action\": \"${action}\",
            \"dedup_key\": \"${dedup_key}\"
        }"
    fi

    curl -s -X POST \
        -H "Content-Type: application/json" \
        "https://events.pagerduty.com/v2/enqueue" \
        -d "$payload"
}

pd_change_event() {
    local summary="$1"
    local source="${2:-deployment}"
    local custom_details="${3:-{}}"

    curl -s -X POST \
        -H "Content-Type: application/json" \
        "https://events.pagerduty.com/v2/change/enqueue" \
        -d "{
            \"routing_key\": \"${PD_ROUTING_KEY}\",
            \"payload\": {
                \"summary\": \"${summary}\",
                \"source\": \"${source}\",
                \"timestamp\": \"$(date -u +%Y-%m-%dT%H:%M:%S.000Z)\",
                \"custom_details\": ${custom_details}
            }
        }"
}

Parallel Execution

# Send multiple independent events in parallel
{
    pd_event "trigger" "disk-full-srv01" "Disk usage >90% on srv01" "critical" "monitoring" &
    pd_event "trigger" "high-cpu-srv02" "CPU usage >95% on srv02" "error" "monitoring" &
    pd_change_event "Deployed api-service v2.3.1" "deployment-pipeline" &
}
wait

Anti-Hallucination Rules

NEVER assume routing keys, dedup keys, or service configurations. ALWAYS validate first.

Phase 1: Validation

#!/bin/bash
echo "=== Validate Routing Key ==="
# Send a test event with acknowledge to verify key works
TEST_RESULT=$(pd_event "trigger" "test-$(date +%s)" "Integration test event" "info" "validation")
STATUS=$(echo "$TEST_RESULT" | jq -r '.status')
DEDUP=$(echo "$TEST_RESULT" | jq -r '.dedup_key')

if [ "$STATUS" = "success" ]; then
    echo "Routing key valid. Status: $STATUS"
    # Immediately resolve the test event
    pd_event "resolve" "$DEDUP" | jq -r '.status'
else
    echo "ERROR: Invalid routing key. Response: $(echo "$TEST_RESULT" | jq -r '.message')"
fi

Common Operations

Trigger Alert Events

#!/bin/bash
SUMMARY="${1:?Alert summary required}"
SEVERITY="${2:-error}"
SOURCE="${3:-cloudskills}"
DEDUP_KEY="${4:-$(echo "$SUMMARY" | md5sum | cut -d' ' -f1)}"

echo "=== Triggering Alert ==="
RESULT=$(pd_event "trigger" "$DEDUP_KEY" "$SUMMARY" "$SEVERITY" "$SOURCE")
echo "$RESULT" | jq '{status: .status, message: .message, dedup_key: .dedup_key}'

Trigger with Custom Details

#!/bin/bash
echo "=== Triggering Detailed Alert ==="
curl -s -X POST \
    -H "Content-Type: application/json" \
    "https://events.pagerduty.com/v2/enqueue" \
    -d "{
        \"routing_key\": \"${PD_ROUTING_KEY}\",
        \"event_action\": \"trigger\",
        \"dedup_key\": \"${1:?dedup_key required}\",
        \"payload\": {
            \"summary\": \"${2:?summary required}\",
            \"severity\": \"${3:-error}\",
            \"source\": \"${4:-cloudskills}\",
            \"timestamp\": \"$(date -u +%Y-%m-%dT%H:%M:%S.000Z)\",
            \"component\": \"${5:-}\",
            \"group\": \"${6:-}\",
            \"class\": \"${7:-}\",
            \"custom_details\": {
                \"environment\": \"production\",
                \"runbook\": \"https://wiki.example.com/runbooks\"
            }
        },
        \"links\": [{\"href\": \"https://monitoring.example.com\", \"text\": \"View in monitoring\"}],
        \"images\": []
    }" | jq '{status, message, dedup_key}'

Acknowledge & Resolve Events

#!/bin/bash
DEDUP_KEY="${1:?Dedup key required}"
ACTION="${2:-acknowledge}"  # acknowledge or resolve

echo "=== ${ACTION} Event ==="
pd_event "$ACTION" "$DEDUP_KEY" | jq '{status, message, dedup_key}'

Change Events (Deployments)

#!/bin/bash
echo "=== Sending Change Event ==="
pd_change_event \
    "${1:?Change summary required}" \
    "${2:-deployment}" \
    "{\"version\":\"${3:-unknown}\",\"environment\":\"${4:-production}\",\"deployed_by\":\"${5:-automation}\"}" \
    | jq '{status, message}'

Batch Alert Management

#!/bin/bash
echo "=== Batch Resolve Alerts ==="
DEDUP_KEYS="${@:?At least one dedup_key required}"

for key in $DEDUP_KEYS; do
    pd_event "resolve" "$key" &
done
wait

echo "Resolved $(echo "$DEDUP_KEYS" | wc -w | tr -d ' ') alerts"

Output Format

Present results as a structured report:

Managing Pagerduty Events Report
════════════════════════════════
Resources discovered: [count]

Resource       Status    Key Metric    Issues
──────────────────────────────────────────────
[name]         [ok/warn] [value]       [findings]

Summary: [total] resources | [ok] healthy | [warn] warnings | [crit] critical
Action Items: [list of prioritized findings]

Target ≤50 lines of output. Use tables for multi-resource comparisons.

Counter-Rationalizations

Common Pitfalls

• Events API vs REST API: Events API (events.pagerduty.com) is for sending alerts — REST API (api.pagerduty.com) is for management
• Routing key vs API key: Events API uses routing/integration keys — NOT the REST API token
• Dedup key: Same dedup_key groups events into one alert — use consistent keys for deduplication
• Severity levels: Only critical, error, warning, info — case-sensitive, no other values
• Rate limits: Events API is rate-limited — 120 events/min per routing key
• Change events: Different endpoint (/v2/change/enqueue) — do NOT send to /v2/enqueue
• Response codes: 202=accepted, 400=invalid payload, 429=rate limited — always check status
• No query API: Events API is fire-and-forget — use REST API to query resulting incidents