Oncall Weekly Telemetry Report

OCE Weekly Report — router

This skill produces the weekly Android on-call telemetry reports. It is a router: it resolves the reporting window, decides which app playbook(s) to run, and stitches the results together into a combined index. All app-specific analysis lives in the playbooks — read the one(s) you need after resolving the mode.

Do not read both playbooks into one context when running both. Run them as two parallel sub-agents (see § Orchestration). Each playbook is large by design; interleaving them degrades both reports and risks cross-contaminating the two apps' incompatible Kusto conventions.

Mode selection

Default is both. Infer the mode from the request; only ask if genuinely ambiguous.

Optional flags the user may add: --skip-crashes (Authenticator only — skips the App Center crash layer, which needs a secret), --end YYYY-MM-DD (see § Reporting window).

Reporting window (shared by both apps)

⚠️ Do NOT ask the user for a reporting date by default. Both reports use the same rolling 7-day window ending at start-of-day UTC on the invocation day, so the most recent complete days are always captured and the two reports are directly comparable. assets/scripts/bootstrap-report.ps1 computes and stamps the window automatically. The previous "confirm the Sunday bucket with the user" flow silently produced stale windows (a Thursday run emitted a 4-day partial window and dropped the last complete week) — that failure mode is what this section explicitly guards against.

1. Reporting window — do NOT ask. Run bootstrap-report.ps1; it resolves the window silently and prints:

   Resolved reporting window (UTC):   # example values for a run on 2026-07-15
     Last 7 days:   2026-07-08 -> 2026-07-15  (exclusive upper bound)
     Baseline:      2026-07-01 -> 2026-07-08
     60-day trend:  2026-05-16 -> 2026-07-15  (literal 60d ending today; chart includes current partial week)
     Trend delta cutoff: weeks < 2026-07-12  (startofweek(curEnd); pass as bucket-trends.js --end)
   

   These dates are stamped into each report's <title>, <div class="meta">, and Generated banner during bootstrap — you do not hand-edit them.

   In both mode, resolve the window ONCE and pass the same -EndDate to both bootstrap calls. Letting each playbook resolve its own window independently is how the two reports drift onto different days when a run straddles UTC midnight.

   Override only when the user explicitly requests a non-default window. Signals: "the week of X", "as of last Friday", "the report from three weeks ago", "in-progress data", "just today's numbers". Then use:

   .\bootstrap-report.ps1 -App broker -EndDate 2026-07-02   # e.g. reproduce the report as of Jul 2
   

   -EndDate is the exclusive upper bound of the current window (curEnd); the script derives curStart = curEnd - 7d and prevStart = curEnd - 14d deterministically. -EndDate must be today or earlier — future dates are refused.

   Supported override = -EndDate only (shifts the window end). The 7-day primary span, the 7-day baseline, and the 60-day trend form a fixed frame that moves rigidly with -EndDate. There is no custom start-date and no arbitrary-span flag — a request like "last 30 days" or "from Jun 1 to Jun 20" is not expressible via a single parameter. To produce a longer or custom span you must edit the KQL window placeholders (<CUR_START> / <CUR_END> / <PREV_START>; the baseline end is always <CUR_START>, so there is no separate <PREV_END> token) by hand. If the user asks for a non-7-day span, say so up front rather than silently emitting a 7-day report.

2. Comparison baseline — auto-computed as the immediately-prior 7 days ([curEnd - 14d, curEnd - 7d)). No user input.

3. 60-day trend window — auto-computed as the literal last 60 days ending today ([curEnd - 60d, curEnd)), so both bounds move with -EndDate. Trend sections are Sun-Sat weekly-bucketed (Kusto startofweek() is Sunday-aligned) because the trend needs stable weekly denominators, but the final bar is the current in-progress (partial) week — the chart ends today. Regression/improvement delta classification is still computed on complete weeks only (bucket-trends.js --end=startofweek(curEnd) --include-partial-end); a partial week as "last" would read as a fake −99% improvement, so it is charted but excluded from the delta math.

Kusto note (weekly-bucketed queries only): startofweek() is Sunday-aligned, so startofweek('2026-05-09') == 2026-05-03T00:00:00Z. When authoring weekly-bucketed queries, verify by printing the distinct week values from your first query. Off-by-one-week is the #1 silent error in weekly-bucket queries.

Outputs

All outputs land in $env:USERPROFILE\android-oce-reports\ — outside the workspace, so reports can never be committed accidentally.

<curEnd> is YYYY-MM-DD, the end-date of the rolling 7-day window. Raw KQL payloads are cached under _data/<app>-<curEnd>/ so each report is reproducible; folders older than 60 days are pruned automatically by bootstrap-report.ps1.

The combined index (both mode only)

After both reports validate, build the index:

.\.github\skills\oncall-weekly-telemetry-report\assets\scripts\build-index.ps1 -EndDate <curEnd>

It reads the headline KPI tiles out of both finished reports and emits a single one-page digest with links to each. It is a digest, not an analysis — do not write new findings into it that aren't already in one of the two reports. If a cross-app finding matters (e.g. an Authenticator Broker-API responsiveness spike that maps to a Broker error code), write it in both reports and let the index link them.

Orchestration (both mode)

Run the two playbooks in parallel as background sub-agents, not sequentially. Serially, both roughly doubles an already-long run; in parallel it costs about as much wall-clock as one report.

1. Resolve the window once (bootstrap-report.ps1 -App broker prints it; capture curEnd).
2. Bootstrap both report files up front with the same -EndDate, so neither agent can drift:

   $S = '.\.github\skills\oncall-weekly-telemetry-report\assets\scripts'
   & "$S\bootstrap-report.ps1" -App broker  -EndDate <curEnd>
   & "$S\bootstrap-report.ps1" -App authapp -EndDate <curEnd>
   

3. Launch two general-purpose background agents. Give each the full context it needs — the playbook path, the resolved window dates, its bootstrapped report path, and its _data folder. Require each agent to read both its playbook and § Shared hard rules in this file before writing any HTML. Instruct each to do the work itself (not to advise), and to run its own validator before reporting back.
4. Wait for both. Do not start the index until both validators pass.
5. Run build-index.ps1.
6. Report the three file paths to the user in chat. Do not paste report contents into chat.

If one agent fails, still publish the other report and say plainly in chat which app failed and why — a half-delivered rotation report beats a blocked one.

Shared assets

App-specific assets (queries, cheatsheets, report templates) are listed in each playbook.

Shared hard rules

These apply to both reports. App-specific hard rules (HLL device counting, TDigest percentiles, Merge* helpers, the Originator pre-check, Authenticator *DCount columns, volume floors) live in the playbooks — and they are not interchangeable. The Broker's "never sum(countDevices)" rule is actively wrong on the Authenticator side, where sum(SucceededDCount) is the correct idiom. Never carry a convention across the two playbooks.

• Never carry a numeric telemetry value forward between runs. Every KPI, table cell, delta %, device/request count, sparkline point, and verdict number must be re-pulled from Kusto for this run — never copied from a previous report, from a checkpoint/summary, from notes, or from memory. Telemetry shifts between runs and stale numbers read as fabricated. Near-miss precedent: a no_tokens_found count was about to be carried as ~23.7M when the actual current-window value was ~4.86M — a ~5× error that only the re-pull caught. If a number isn't backed by a query result file in this run's _data/<app>-<curEnd>/, it does not go in the report.
• Never hardcode the "Generated" date. It is the run date in UTC, auto-stamped by bootstrap-report.ps1. If you rebuild a body programmatically, derive it live with a UTC-date formatter (new Date().toISOString().slice(0,10) in Node, [datetime]::UtcNow.ToString('yyyy-MM-dd') in PowerShell) — never paste a literal.
• ⚠️ UTF-8 trap — DO NOT use PowerShell @'...'@ heredocs to compose HTML containing emojis, em-dashes, arrows, or middle dots. PowerShell silently strips multi-byte UTF-8 characters when piping heredocs to Set-Content / Out-File. Use Node (fs.writeFileSync), [IO.File]::WriteAllText($path, $text, [System.Text.UTF8Encoding]::new($false)), or explicit Unicode-pair literals ([char]0xD83D + [char]0xDCCA for 📊). The validator's U+FFFD check catches the worst case (mojibake) but cannot detect characters silently stripped to nothing.
• Never use regex to bulk-edit balanced HTML. This skill has burned twice on regex strip scripts that ate matched-pair </div> closes, producing inception-style nested-callout bugs that take a depth-tracking script to find. Make targeted, single-occurrence string replacements (with explicit before/after context) or rewrite the affected block end-to-end.
• No shorthand in user-facing text. All UI text — callouts, table headers, KPI labels, verdicts, badges — says devices and requests, never devs / reqs. Internal variable, column, and file names can stay short.
• A moved metric is a question, not a verdict. Never publish a regression verdict without the app's diagnostic ladder having been walked (Broker: Originator pre-check + dim slicing; Authenticator: volume floor + rate normalisation + error-reason decomposition).
• Filename collision rule. If a report already exists for the same end-date, do not silently overwrite. Open it, list its top-3 findings, and explicitly state in chat what changed in the new data before regenerating. A second run on the same window without a delta is wasted work.
• Do not create a separate Markdown summary of either report — the HTML is the deliverable.
• Do not commit any report. They live in $env:USERPROFILE\android-oce-reports\ precisely so they can't be staged accidentally.

Router checklist

• Mode resolved (both default) and stated in chat before work starts.
• Window resolved once and the same -EndDate passed to every bootstrap call.
• Each selected playbook read in full by the agent that owns it.
• In both mode, the two playbooks ran as parallel sub-agents, not interleaved in one context.
• Each report passed its own validate-report.ps1 -App <app> run.
• oce-index-<curEnd>.html built (in both mode) after both validators passed.
• File paths reported in chat; no report content pasted into chat; nothing committed.