1---2name: grafana-sync-orchestrator3description: Use when tasks require grafana sync orchestrator with credential-aware preflight, deterministic execution, validation gates, and handoff-ready artifacts.4---56# Grafana Sync Orchestrator78## Quick Reference9| Field | Value |10| --- | --- |11| Skill ID | `632` |12| Provider | `Grafana` |13| Operation | `Sync Orchestrator` |14| Domain | `Observability and incident platforms` |15| Runtime archetype | `sync-orchestration-engine` |16| Core method | `cursor-based reconciliation` |17| Primary artifact | `grafana-reconciliation-bundle` |18| Routing tag | `grafana:sync-orchestrator` |19| Mutating | `yes` |20| Release cycles | `2` |2122## Why This Skill Exists23We need this skill because Grafana workflows degrade when auth, schema, and side-effect handling drift when integrations are run ad hoc. This specific skill turns Grafana Sync Orchestrator into a deterministic, auth-checked workflow for schedules and reconciles two-way sync for grafana records..2425## Trigger Checklist26- [ ] The task explicitly requires `Grafana Sync Orchestrator` rather than generic brainstorming.27- [ ] The provider tenant, workspace, or environment is known before execution begins.28- [ ] Credential reuse has been checked before asking for new secrets.29- [ ] Success criteria, side effects, and handoff owner are clear.30- [ ] If the run mutates provider state, the relevant approval gates are available.3132## Auth & Access Profile33| Field | Value |34| --- | --- |35| External auth required | `yes` |36| API key likely required | `yes` |37| Protocols | `HTTPS/REST`, `webhook callbacks` |38| Mutating | `yes` |39| Webhook capable | `no` |4041| Auth Mode | Kind | Env Hints | Validation |42| --- | --- | --- | --- |43| API key or account secret | `api_key` | `GRAFANA_API_KEY` | Validate the key with a lightweight authenticated request and confirm environment or mode (sandbox vs production). |44| Access token or personal access token | `token` | `GRAFANA_TOKEN`, `GRAFANA_ACCESS_TOKEN` | Validate the token with the smallest read-only endpoint that proves scope and tenancy. |4546## Inputs (contract)47| Input | Type | Required | Source |48| --- | --- | --- | --- |49| source cursor | signal | yes | operator or upstream tool |50| destination state | signal | yes | operator or upstream tool |51| conflict policy | signal | yes | operator or upstream tool |5253## Outputs (contract)54| Output | Type | Guaranteed | Consumer |55| --- | --- | --- | --- |56| grafana-reconciliation-bundle | structured-artifact | yes | next workflow or operator |57| grafana-reconciliation-bundle-scorecard | scorecard | yes | reviewer |58| grafana-reconciliation-bundle-handoff | handoff-packet | yes | downstream owner |5960## Step-by-Step Implementation Guide611. Define the systems, entities, and conflict policy for Grafana Sync Orchestrator, then checkpoint the last known good cursor on both sides of Grafana.622. Validate credential reuse and rate-limit posture for every participating endpoint before the sync window opens.633. Specify delta-fetch, reconciliation, and idempotency contracts for schedules and reconciles two-way sync for grafana records., including replay safety and backfill rules.644. Process deltas in deterministic batches, checkpoint after every write phase, and quarantine records that exceed conflict thresholds.655. Exercise simulation and regression suites that cover stale cursors, duplicate deliveries, and asymmetric provider failures.666. Publish a reconciliation bundle with drift metrics, conflict counts, and the next checkpoint required for autonomous continuation.6768## Operational Runbook69### Preflight70- Verify both source and destination auth contexts and confirm the active cursor/checkpoint.71- Agree on the conflict policy before any mutation is emitted to the destination system.7273### Execution74- Fetch deltas deterministically, checkpoint every batch, and preserve replay inputs.75- Route unresolved conflicts into a human review queue instead of guessing.7677### Recovery78- Freeze the sync window when reconciliation diverges from the checkpointed baseline.79- Replay from the last stable cursor only after the conflict policy is corrected.8081### Handoff82- Return checkpoint state, drift summary, and unresolved conflicts with next owners.83- Document whether the destination is caught up, backfilled, or paused pending review.8485## Validation Gates & Test Matrix86| Gate | Purpose | On Fail |87| --- | --- | --- |88| auth-preflight | Validate credential presence, scope, and environment before work begins. | block execution |89| schema-contract-check | Ensure required signals and payload shapes remain valid. | quarantine and request correction |90| policy-approval-check | Verify the declared approval gates before mutating or publishing state. | pause or route to human review |91| reliability-check | Confirm retries, rollback, and checkpoint readiness. | rollback or fail closed |9293- Required validation suites: `unit`, `integration`, `simulation`, `regression-baseline`9495## Failure Modes & Recovery Playbook96| Code | Trigger | Action |97| --- | --- | --- |98| `E_CURSOR_GAP` | Missing or corrupted source/destination checkpoint. | Freeze sync and require a manually selected replay point. |99| `E_CONFLICT_BURST` | Delta batches exceed safe conflict thresholds. | Block writeback, emit a conflict packet, and route to review. |100| `E_AUTH_CONTEXT` | Either side of the sync lacks valid credentials. | Stop the sync window and revalidate both auth contexts before replay. |101102## Tool Call Implementation103- Reuse existing credentials first. Check environment variables, secure stores, and active sessions before prompting.104- Start with the smallest authenticated read or validation call that proves identity and scope.105- Preserve request, response, and approval traces in `grafana-reconciliation-bundle` so downstream owners do not need to rediscover context.106- If any auth, contract, or approval gate fails, halt execution and attach remediation guidance instead of guessing.107108## Credential Reuse Policy109- Reuse valid provider credentials by default and prefer tenant-scoped sessions over newly created secrets.110- Prompt for credentials only when they are missing, invalid, expired, or point at the wrong environment.111- For webhook flows, validate the signing secret against a known sample before accepting live traffic.112113## Guardrails114- quality: Checkpoint every delta batch before applying destination writes. (`cursor-checkpoint`)115- reliability: Reconcile from the last stable cursor after failures or drift. (`cursor-replay`)116- compliance: Escalate unresolved conflicts instead of auto-merging beyond policy. (`conflict-review-queue`)117- compliance: Validate provider key or secret mode (sandbox vs production) before any mutating execution. (`credential-mode-check`)118119## Acceptance Checklist120- [ ] Credential preflight and scope validation completed successfully.121- [ ] Required validation suites ran and all fail-closed gates passed.122- [ ] grafana-reconciliation-bundle, scorecard, and handoff packet were produced.123- [ ] Any mutations, approvals, or rollbacks are reflected in the artifact bundle.124125## Anti-Patterns126- Do not ask for new credentials before checking reusable auth context.127- Do not skip the read-only or dry-run validation step for mutating work.128- Do not proceed when approval gates, signing secrets, or rollback checkpoints are missing.129- Do not hand off partial or ambiguous provider state as complete.130131## Handoff Contract132- **Produces:** `grafana-reconciliation-bundle`, execution scorecard, approval trace, and next actions.133- **Consumes:** `source cursor`, `destination state`, `conflict policy`.134- **Readiness rule:** release only after auth, contract, approval, and reliability gates all pass.135- **Downstream hint:** route to `grafana:sync-orchestrator` consumers with approval and credential context attached.136137## Observability & Continuous Improvement138- SLO: >=99.9% successful runs per 7-day window139- Error budget: <=0.1% critical failures per 7-day window140- Alert triggers:141- credential validation failures exceed baseline142- schema or contract regressions persist for two consecutive runs143- critical posture or rollback events exceed tolerance144- Primary outcome metric: `sync lag`145- Secondary metrics: `reconciliation completion rate`, `conflict resolution accuracy`146- Review cadence: `daily`