Report Migration Coverage

Report migration coverage

Audience: operators of the published obs-migrate CLI (PyPI/uvx), using public docs and their real source + Elastic/Kibana — not a repo lab harness.

Goal: compose a shareable coverage summary from artifacts a completed obs-migrate migrate run already wrote on disk — read-only; do not re-run migration or touch any cluster.

Prerequisites (install)

These skills help operators of the published CLI (not a repo checkout). If obs-migrate is missing or doctor is not Ready, follow install-obs-migrate first — that skill owns PyPI/uvx/pip, extras, and Python/uv gotchas. Do not invent alternate install commands here.

uvx --from 'elastic-observability-migration[all]' obs-migrate doctor
# After a persistent install, the same check is: obs-migrate doctor

Source/Elastic credentials: connect-to-o11y-source (and your env exports).

Inputs (artifact table)

Workflow

1. Locate the output dir — the --output-dir from the user's migrate run (or ask which run they mean if several exist).
2. Read the headline verdict — open <output-dir>/dashboards/migration_summary.md for the human-readable scorecard and must-fix worklist. Prefer quoting its scorecard % when presenting to a manager — it already denominates over translated widgets.
3. Read the counts — open <output-dir>/dashboards/migration_manifest.json and read the summary object.
4. Compute coverage — from summary (clean/warning key names differ by source — pick the right pair for the run):
   • Grafana: clean = summary.migrated, warnings = summary.migrated_with_warnings.
   • Datadog: clean = summary.ok, warnings = summary.warning.
   • Both: summary.requires_manual, summary.not_feasible; Datadog also summary.blocked.
   • Coverage % — prefer the Migrated row in migration_summary.md when present. If computing from JSON: clean / (summary.ok|migrated + warnings + requires_manual + not_feasible + blocked). Do not blindly divide by summary.panels on Datadog: summary.panels can include skipped structural/group widgets that are not in the scorecard buckets (count status=="skipped" in panels[] and call them out separately).
   • Needs review = warnings count + summary.requires_manual.
   • Blocked = summary.not_feasible (+ summary.blocked on Datadog).
5. Check scope — read <output-dir>/run_summary.json to see whether dashboards and/or alerts actually ran (ran.dashboards, ran.alerts) and how many were in scope (dashboards.total, alerts.total).
6. Optional appendix — if the audience cares about field/schema gaps, run obs-migrate schema-report --artifact-dir <output-dir>/dashboards --output schema_change_report.md and attach or link the table.
7. Assemble the shareable summary — one short narrative (verdict + coverage % + needs-review + blocked counts + alert scope note) backed by the artifact paths above.

Manual-effort estimate

Give a rough bucket estimate, not false precision:

• requires_manual — moderate rebuild each (partial translation; human must finish or replace the panel).
• not_feasible (+ blocked on Datadog) — full manual rebuild each (engine could not produce a usable panel).
• skipped (Datadog) — usually not effort (group/structural containers); do not bill them as rebuilds unless the user expected a Kibana panel for that widget.

State the counts from summary (and skipped from panels[] when relevant) and multiply by the bucket above (e.g. "12 requires_manual ≈ moderate rework each; 4 not_feasible ≈ full rebuild each"). Avoid implying exact person-hours unless the user supplies their own velocity.

Degrade gracefully / Honest limits

• exit 0 does not mean every panel is perfect; warnings and blocked panels are listed, never hidden. Trust migration_summary.md and migration_manifest.json, not the exit code alone.
• Alerts coverage depends on what ran. If the migrate command used --assets dashboards only, run_summary.json shows ran.alerts: false — do not claim alert migration coverage without reading that file.
• Datadog and Grafana use different clean/warning keys in summary. Grafana writes summary.migrated and summary.migrated_with_warnings; Datadog writes summary.ok and summary.warning (not migrated / migrated_with_warnings) and adds summary.blocked. summary.requires_manual and summary.not_feasible are shared — read the JSON for the source you are reporting on.
• This skill does not prove panels render correctly — empty uploaded panels may be missing telemetry, not a translation bug. Numerical proof is validate-side-by-side; per-panel gap explanations are explain-migration-gaps; UI/runtime emptiness after upload is debug-uploaded-kibana-dashboard / grafana-validate-uploaded.

See also

• install-obs-migrate — install/doctor when the CLI is missing or not Ready.
• explain-migration-gaps skill — turn each panel reason into manual rebuild guidance.
• validate-side-by-side skill — prove migrated panels match source numerically.
• debug-uploaded-kibana-dashboard skill — empty/wrong UI after upload.
• https://github.com/elastic/observability-migration-platform/blob/main/docs/command-contract.md — artifact paths and migrate flags for the installed version.