Zava Learning — Connectivity / Edge Incident Runbook
Resource Group: @@RG@@. Public entry: Application Gateway -> learner-portal (Container App,
internal ingress) -> course-api / assessment-api (environment-internal). App Insights
cloud_RoleName values: learner-portal, course-api, assessment-api.
Diagnose root cause from telemetry and configuration, then remediate within the boundary below.
Do NOT guess the cause from the alert name — the alert is symptom-only by design.
Trace the path, hop by hop
- Application Gateway — backend health (
az network application-gateway show-backend-health),
probe path/host, HTTP settings. A probe pointed at a path the portal doesn't serve marks the
backend unhealthy and yields 502s.
- NSG on the Container Apps subnet — list effective rules. A higher-priority DENY can beat a
lower-priority ALLOW (priority inversion) and silently block App Gateway -> apps.
- Container Apps internal load balancer / ingress — revision health, replica counts.
- APIs — are
course-api / assessment-api answering and healthy?
Use the built-in network troubleshooting skills (network_connectivity_troubleshoot,
application_gateway_troubleshoot, load_balancer_troubleshoot, network_topology_mapper) to go deep
on any hop. Filter App Insights/LAW queries by the relevant cloud_RoleName.
Permitted autonomous actions
- Neutralize a blocking NSG rule with a non-destructive update, not a delete: run
az network nsg rule update ... --access Allow (or raise the DENY rule's --priority above the
ALLOW). The write tool restricts delete/remove, and an update achieves the same effect — so
never reach for az network nsg rule delete.
- Correct an Application Gateway probe path / HTTP settings back to a healthy configuration.
- Restart a Container Apps revision.
Azure CLI usage (avoid avoidable command failures)
- Do not pass
-o/--output or --query to RunAzCliReadCommands. The read tool already
returns JSON — adding -o json, -o table, or a --query projection makes the command fail with
a generic "Unknown error occurred." Run the plain command (e.g. az network nsg rule list --nsg-name ... --include-default) and pick out the fields you need from the JSON in your reasoning.
- If any read still returns "Unknown error occurred," just retry the plain command once — the
first tool call in a session can fail transiently. Do not conclude the resource is broken.
- Always pass
--subscription and prefer resource IDs to avoid ambiguity. Consult GetAzCliHelp
before an unfamiliar write flag rather than guessing syntax.
Incident communication (PagerDuty)
Record the request-path diagram and your diagnostic notes for the incident record. PagerDuty
acknowledgement, status/summary notes, and resolution are owned by the pagerduty-incident-update
skill.
Code & change management
- For an Infrastructure-as-Code root cause, the infra lives under
infra/ in @@REPO@@
(the NSG is defined in infra/modules/network.bicep). After the live mitigation, the durable
fix is delivered as a GitHub pull request by the pr-delivery skill and recorded as a Change
Request by servicenow-change-management.
Out of scope (require human approval)
- VNet address-space changes, subnet deletion, IAM modifications, App Gateway SKU/tier changes.
Verification
Re-check the hop you changed, confirm the public endpoint returns 200 on / and /api/quiz/*, and
confirm the alert auto-mitigated.
1---2metadata:3 source: https://skillmd.com/skills/microsoft/connectivity-triage4 api_version: azuresre.ai/v25 kind: Skill6name: connectivity-triage7description: Use for any Zava Learning incident where students cannot reach the platform or actions fail at the network/edge layer — quiz launches failing, portal 5xx, requests timing out, or backends appearing unhealthy. Traces the full request path (Application Gateway -> NSG -> Container Apps internal load balancer -> APIs) from telemetry and Azure config, finds the broken hop, and remediates within the permitted-action boundary.8tools:9 - RunAzCliReadCommands10 - RunAzCliWriteCommands11 - GetAzCliHelp12 - SearchMemory13 - microsoft-learn_microsoft_docs_search14 - microsoft-learn_microsoft_docs_fetch15---1617## Zava Learning — Connectivity / Edge Incident Runbook1819Resource Group: `@@RG@@`. Public entry: Application Gateway -> learner-portal (Container App,20internal ingress) -> course-api / assessment-api (environment-internal). App Insights21`cloud_RoleName` values: `learner-portal`, `course-api`, `assessment-api`.2223Diagnose root cause from telemetry and configuration, then remediate within the boundary below.24Do NOT guess the cause from the alert name — the alert is symptom-only by design.2526## Trace the path, hop by hop271. **Application Gateway** — backend health (`az network application-gateway show-backend-health`),28 probe path/host, HTTP settings. A probe pointed at a path the portal doesn't serve marks the29 backend unhealthy and yields 502s.302. **NSG on the Container Apps subnet** — list effective rules. A higher-priority DENY can beat a31 lower-priority ALLOW (priority inversion) and silently block App Gateway -> apps.323. **Container Apps internal load balancer / ingress** — revision health, replica counts.334. **APIs** — are `course-api` / `assessment-api` answering and healthy?3435Use the built-in network troubleshooting skills (network_connectivity_troubleshoot,36application_gateway_troubleshoot, load_balancer_troubleshoot, network_topology_mapper) to go deep37on any hop. Filter App Insights/LAW queries by the relevant `cloud_RoleName`.3839## Permitted autonomous actions40- **Neutralize a blocking NSG rule with a non-destructive update**, not a delete: run41 `az network nsg rule update ... --access Allow` (or raise the DENY rule's `--priority` above the42 ALLOW). The write tool restricts `delete`/`remove`, and an `update` achieves the same effect — so43 never reach for `az network nsg rule delete`.44- Correct an Application Gateway probe path / HTTP settings back to a healthy configuration.45- Restart a Container Apps revision.4647## Azure CLI usage (avoid avoidable command failures)48- **Do not pass `-o`/`--output` or `--query` to `RunAzCliReadCommands`.** The read tool already49 returns JSON — adding `-o json`, `-o table`, or a `--query` projection makes the command fail with50 a generic "Unknown error occurred." Run the plain command (e.g. `az network nsg rule list51 --nsg-name ... --include-default`) and pick out the fields you need from the JSON in your reasoning.52- If any read still returns "Unknown error occurred," just **retry the plain command once** — the53 first tool call in a session can fail transiently. Do not conclude the resource is broken.54- Always pass `--subscription` and prefer resource IDs to avoid ambiguity. Consult `GetAzCliHelp`55 before an unfamiliar write flag rather than guessing syntax.5657## Incident communication (PagerDuty)58Record the request-path diagram and your diagnostic notes for the incident record. PagerDuty59acknowledgement, status/summary notes, and resolution are owned by the `pagerduty-incident-update`60skill.6162## Code & change management63- For an Infrastructure-as-Code root cause, the infra lives under `infra/` in `@@REPO@@`64 (the NSG is defined in `infra/modules/network.bicep`). After the live mitigation, the durable65 fix is delivered as a GitHub pull request by the `pr-delivery` skill and recorded as a Change66 Request by `servicenow-change-management`.6768## Out of scope (require human approval)69- VNet address-space changes, subnet deletion, IAM modifications, App Gateway SKU/tier changes.7071## Verification72Re-check the hop you changed, confirm the public endpoint returns 200 on `/` and `/api/quiz/*`, and73confirm the alert auto-mitigated.