acedergren OCI Monitoring and Observability - Expert Knowledge
Do NOT load this skill when
Do not load this skill for unrelated general programming, non-Oracle cloud work, or questions covered by a narrower sibling skill.
When the request is only asking to find or install skills, use find-skills instead.
When to Use
Load this skill for: the user asks to "create OCI alarms", "debug missing metrics", "write MQL", "configure Service Connector", or "monitor OCI resources".
Prefer this skill only for its named domain. For broader OCI architecture triage, start with oci/best-practices as the router.
NEVER Do This
NEVER debug "missing metrics" within the first 15 minutes
- Metrics are published every 1–5 minutes
- Processing delay adds another 5–10 minutes
- Total lag from event to visible metric: 10–15 minutes
- Premature debugging creates false investigations
NEVER use = for alarm thresholds with sparse metrics
# WRONG - alarm never fires when metric has data gaps
MetricName[1m].mean() = 0
# RIGHT - handle missing data explicitly
MetricName[1m]{dataMissing=zero}.mean() > 0
NEVER omit the resourceId dimension in metric queries
# WRONG - returns no data (required dimension missing)
CPUUtilization[1m].mean()
# RIGHT - filter by instance OCID
CPUUtilization[1m]{resourceId="<instance-ocid>"}.mean()
Querying without dimensions returns data for ALL resources — usually not what's intended, and rate-limited at 1000 req/min.
NEVER set alarm thresholds without a trigger delay
# BAD - fires on every transient CPU spike (alert fatigue)
CPUUtilization[1m].mean() > 80
# BETTER - fires only on sustained breach
CPUUtilization[5m].mean() > 80
# + set trigger delay: 5 minutes (5 consecutive breaches)
NEVER create alarms without notification destinations
# WRONG - alarm fires but nobody is notified
oci monitoring alarm create ... --destinations '[]'
# RIGHT - always link to a notification topic
oci monitoring alarm create ... --destinations '["<notification-topic-ocid>"]'
Cost impact: undetected production outages = $5,000–50,000+/hour.
NEVER ignore Cloud Guard findings
- Cloud Guard detects misconfigurations before they become incidents
- Wire it: Cloud Guard → Notifications → email/Slack/PagerDuty
- Unresolved findings fail CIS/SOC2/HIPAA audits
Metric Namespace Reference
OCI uses service-specific namespaces — using the wrong namespace returns no data with no error.
| Service | Namespace | Key Metrics |
|---|---|---|
| Compute | oci_computeagent |
CPUUtilization, MemoryUtilization |
| Autonomous DB | oci_autonomous_database |
CpuUtilization, StorageUtilization |
| Load Balancer | oci_lbaas |
HttpRequests, UnHealthyBackendServers |
| Object Storage | oci_objectstorage |
ObjectCount, BytesUploaded |
Common mistake: using oci_compute instead of oci_computeagent — the agent namespace requires the OCI Compute Agent to be running on the instance.
Alarm Missing Data Handling
| Setting | Behavior | Use When |
|---|---|---|
treatMissingDataAsBreaching |
Alarm fires if no data arrives | Critical services (silence = outage) |
treatMissingDataAsNotBreaching |
Alarm silent if no data | Optional or intermittent monitoring |
{dataMissing=zero} in MQL |
Treats gaps as 0 value | Request counters, throughput metrics |
Log Collection Troubleshooting
Logs not appearing in Log Analytics?
│
├─ Is logging enabled on the resource?
│ └─ Compute: is oci-compute-agent running? (systemctl status oracle-cloud-agent)
│ └─ Functions: is logging enabled in function configuration?
│
├─ Is Service Connector configured and ACTIVE?
│ └─ Source: Log Group → Target: Log Analytics
│ └─ Check status: oci sch service-connector get --id <ocid>
│
├─ IAM policy for Service Connector?
│ └─ "Allow any-user to use log-content in tenancy"
│ └─ "Allow service loganalytics to READ logcontent in tenancy"
│ └─ Missing EITHER policy causes silent failure
│
└─ 10–15 minute ingestion lag?
└─ Wait before concluding logs are missing
Metric Query Performance
Unfiltered queries scan ALL resources in compartment — slow and consumes rate limit budget.
# Expensive: scans all instances
CPUUtilization[1m].mean()
# Optimized: filter to specific instance
CPUUtilization[1m]{resourceId='<instance-ocid>'}.mean()
Rate limit: 1000 metric queries/minute per tenancy. Dashboard with many unfiltered widgets can exhaust this.
Progressive Loading Reference
Load references/oci-monitoring-reference.md when:
- Need the complete list of OCI service metric namespaces and metric names
- Writing complex MQL expressions (composites, functions, grouping)
- Implementing composite alarm conditions
- Setting up Log Analytics workspace, APM, or Service Connector Hub in detail
Do NOT load for alarm threshold patterns, namespace gotchas, or log troubleshooting — this file covers those.
Arguments
$ARGUMENTS: Optional user-provided target, path, environment, symptom, or constraint. When empty, infer the narrowest safe scope from the current repository context and ask only if multiple high-impact choices remain.