Datadog Audit Trail
Investigate user activity, configuration changes, access patterns, and compliance evidence using pup audit-logs.
Sub-Skills
| Sub-skill |
Use when |
| security-investigation |
"Who changed X?", "What did this user do?", "Show me deletions in the last 24h" |
| key-compromise |
"Was this API key compromised?", "What did key XYZ do?", "Investigate suspicious key activity" |
| cost-spike-investigation |
"Why did my bill go up?", "What caused this usage spike?", "Investigate LLM cost increase" |
| compliance-report |
"Generate SOC 2 evidence", "PCI audit log", "User provisioning report for auditor" |
| ai-activity-audit |
"What did the AI assistant do?", "Audit MCP tool calls", "AI governance report" |
Prerequisites
pup auth login # OAuth2 (recommended)
# or set DD_API_KEY + DD_APP_KEY with audit_logs_read scope
Commands
# List recent events
pup audit-logs list --from 1h --limit 100
# Search with a query
pup audit-logs search --query "@action:deleted" --from 24h
# JSON output for piping to jq
pup audit-logs search --query "@usr.email:alice@example.com" --from 7d -o json | jq '.data[].attributes'
Event Schema Quick Reference
| Field |
Description |
Example values |
@usr.email |
Actor email |
alice@example.com |
@evt.actor.type |
How action was taken |
USER, API_KEY, SUPPORT_USER |
@action |
Verb |
created, modified, deleted, accessed, login |
@evt.name |
Event category |
Dashboard, Monitor, Authentication, Access Management |
@asset.type |
Resource type |
dashboard, monitor, api_key, role, user |
@asset.id |
Resource identifier |
abc-123 |
@metadata.api_key.id |
API key used (if applicable) |
key_abc123 |
@metadata.app_key.id |
App key used (if applicable) |
app_abc123 |
@network.client.ip |
Client IP address |
1.2.3.4 |
@network.client.geoip.country.name |
Country |
United States |
@network.client.geoip.as.name |
ASN name |
Amazon.com |
@http.url_details.path |
API endpoint path |
/api/v1/dashboard/xyz |
Search Syntax
Same Lucene-style syntax as Log Explorer:
| Query |
Meaning |
@evt.name:Dashboard |
Exact field match |
@action:deleted |
Action filter |
@usr.email:alice@example.com |
Specific user |
@evt.name:Monitor AND @action:modified |
Compound |
-@action:deleted |
Negation |
@usr.email:* |
Field exists |
@network.client.ip:1.2.3.4 |
IP filter |
Retention
Default retention is 90 days. If querying beyond 90 days, archive to S3/GCS/Azure Blob must be configured. Always check whether the requested time window falls within retention before running a query.
Troubleshooting
| Problem |
Cause |
Fix |
| 403 Forbidden |
Missing audit_logs_read scope |
Add scope to app key in Datadog UI |
| Empty results |
Time window outside retention |
Check archive config; default max is 90 days |
| Timeout |
Query too broad |
Narrow time window or add more filters |
| No IP data |
Internal action or pre-enrichment event |
Not all events have geo data |
References
1---2name: dd-audit3description: Audit Trail investigations - who changed what, key compromise, cost spike root cause, compliance evidence (SOC 2/PCI), and AI activity auditing.4---5
6# Datadog Audit Trail
7
8Investigate user activity, configuration changes, access patterns, and compliance evidence using `pup audit-logs`.
9
10## Sub-Skills
11
12| Sub-skill | Use when |
13|-----------|----------|
14| **security-investigation** | "Who changed X?", "What did this user do?", "Show me deletions in the last 24h" |
15| **key-compromise** | "Was this API key compromised?", "What did key XYZ do?", "Investigate suspicious key activity" |
16| **cost-spike-investigation** | "Why did my bill go up?", "What caused this usage spike?", "Investigate LLM cost increase" |
17| **compliance-report** | "Generate SOC 2 evidence", "PCI audit log", "User provisioning report for auditor" |
18| **ai-activity-audit** | "What did the AI assistant do?", "Audit MCP tool calls", "AI governance report" |
19
20## Prerequisites
21
22```bash
23pup auth login # OAuth2 (recommended)
24# or set DD_API_KEY + DD_APP_KEY with audit_logs_read scope
25```
26
27## Commands
28
29```bash
30# List recent events
31pup audit-logs list --from 1h --limit 100
32
33# Search with a query
34pup audit-logs search --query "@action:deleted" --from 24h
35
36# JSON output for piping to jq
37pup audit-logs search --query "@usr.email:alice@example.com" --from 7d -o json | jq '.data[].attributes'
38```
39
40## Event Schema Quick Reference
41
42| Field | Description | Example values |
43|-------|-------------|----------------|
44| `@usr.email` | Actor email | `alice@example.com` |
45| `@evt.actor.type` | How action was taken | `USER`, `API_KEY`, `SUPPORT_USER` |
46| `@action` | Verb | `created`, `modified`, `deleted`, `accessed`, `login` |
47| `@evt.name` | Event category | `Dashboard`, `Monitor`, `Authentication`, `Access Management` |
48| `@asset.type` | Resource type | `dashboard`, `monitor`, `api_key`, `role`, `user` |
49| `@asset.id` | Resource identifier | `abc-123` |
50| `@metadata.api_key.id` | API key used (if applicable) | `key_abc123` |
51| `@metadata.app_key.id` | App key used (if applicable) | `app_abc123` |
52| `@network.client.ip` | Client IP address | `1.2.3.4` |
53| `@network.client.geoip.country.name` | Country | `United States` |
54| `@network.client.geoip.as.name` | ASN name | `Amazon.com` |
55| `@http.url_details.path` | API endpoint path | `/api/v1/dashboard/xyz` |
56
57## Search Syntax
58
59Same Lucene-style syntax as Log Explorer:
60
61| Query | Meaning |
62|-------|---------|
63| `@evt.name:Dashboard` | Exact field match |
64| `@action:deleted` | Action filter |
65| `@usr.email:alice@example.com` | Specific user |
66| `@evt.name:Monitor AND @action:modified` | Compound |
67| `-@action:deleted` | Negation |
68| `@usr.email:*` | Field exists |
69| `@network.client.ip:1.2.3.4` | IP filter |
70
71## Retention
72
73Default retention is **90 days**. If querying beyond 90 days, archive to S3/GCS/Azure Blob must be configured. Always check whether the requested time window falls within retention before running a query.
74
75## Troubleshooting
76
77| Problem | Cause | Fix |
78|---------|-------|-----|
79| 403 Forbidden | Missing `audit_logs_read` scope | Add scope to app key in Datadog UI |
80| Empty results | Time window outside retention | Check archive config; default max is 90 days |
81| Timeout | Query too broad | Narrow time window or add more filters |
82| No IP data | Internal action or pre-enrichment event | Not all events have geo data |
83
84## References
85
86- [Audit Trail API](https://docs.datadoghq.com/api/latest/audit/)
87- [Audit Trail documentation](https://docs.datadoghq.com/account_management/audit_trail/)
88- [Search syntax](https://docs.datadoghq.com/logs/explorer/search_syntax/)