Azure Openai Report Builder

Azure OpenAI Report Builder

Quick Reference

Why This Skill Exists

We need this skill because Azure OpenAI workflows degrade when auth, schema, and side-effect handling drift when integrations are run ad hoc. This specific skill turns Azure OpenAI Report Builder into a deterministic, auth-checked workflow for generates operational and kpi reports from azure openai activity..

Trigger Checklist

• The task explicitly requires Azure OpenAI Report Builder rather than generic brainstorming.
• The provider tenant, workspace, or environment is known before execution begins.
• Credential reuse has been checked before asking for new secrets.
• Success criteria, side effects, and handoff owner are clear.
• If the run mutates provider state, the relevant approval gates are available.

Auth & Access Profile

Inputs (contract)

Outputs (contract)

Step-by-Step Implementation Guide

1. Define the report audience, metric definitions, and freshness threshold for Azure OpenAI Report Builder, then confirm which Azure OpenAI entities are in scope.
2. Validate credential reuse and ensure the query window matches the reporting cadence before any render starts.
3. Implement generates operational and kpi reports from azure openai activity. with deterministic metric synthesis, template versioning, and publication gating.
4. Capture metric lineage, freshness timestamps, and template identifiers in the report bundle so the output remains auditable.
5. Run regression checks that cover stale data, metric drift, and template mismatch before publishing the report.
6. Publish the report bundle only after freshness and correctness gates pass, along with any interpretive notes required for downstream readers.

Operational Runbook

Preflight

• Validate the metric definitions, time window, and publication audience before building the report.
• Confirm the source data contract and output template version.

Execution

• Compute metrics deterministically and preserve intermediate query identifiers.
• Render the report only after freshness and completeness checks pass.

Recovery

• Hold publication when metrics are stale, incomplete, or fail reconciliation checks.
• Rebuild from the last known good query window after correcting the data issue.

Handoff

• Return the report bundle, metric lineage, and publication decision.
• Call out any missing windows, excluded entities, or operator notes needed to interpret the report.

Validation Gates & Test Matrix

• Required validation suites: unit, integration, regression-baseline

Failure Modes & Recovery Playbook

Tool Call Implementation

• Reuse existing credentials first. Check environment variables, secure stores, and active sessions before prompting.
• Start with the smallest authenticated read or validation call that proves identity and scope.
• Preserve request, response, and approval traces in azure-openai-report-bundle so downstream owners do not need to rediscover context.
• If any auth, contract, or approval gate fails, halt execution and attach remediation guidance instead of guessing.

Credential Reuse Policy

• Reuse valid provider credentials by default and prefer tenant-scoped sessions over newly created secrets.
• Prompt for credentials only when they are missing, invalid, expired, or point at the wrong environment.
• For webhook flows, validate the signing secret against a known sample before accepting live traffic.

Guardrails

• quality: Hold report publication when metrics drift from declared definitions or freshness thresholds. (metric-reconcile+freshness-check)
• reliability: Preserve query lineage and output template versions for every report build. (lineage-capture)
• cost: Reuse cached intermediate results when they remain inside freshness tolerance. (query-cache-policy)
• compliance: Validate provider key or secret mode (sandbox vs production) before any mutating execution. (credential-mode-check)

Acceptance Checklist

• Credential preflight and scope validation completed successfully.
• Required validation suites ran and all fail-closed gates passed.
• azure-openai-report-bundle, scorecard, and handoff packet were produced.
• Any mutations, approvals, or rollbacks are reflected in the artifact bundle.

Anti-Patterns

• Do not ask for new credentials before checking reusable auth context.
• Do not skip the read-only or dry-run validation step for mutating work.
• Do not proceed when approval gates, signing secrets, or rollback checkpoints are missing.
• Do not hand off partial or ambiguous provider state as complete.

Handoff Contract

• Produces: azure-openai-report-bundle, execution scorecard, approval trace, and next actions.
• Consumes: report scope, metric definitions, presentation template.
• Readiness rule: release only after auth, contract, approval, and reliability gates all pass.
• Downstream hint: route to azure-openai:report-builder consumers with approval and credential context attached.

Observability & Continuous Improvement

• SLO: >=99.9% successful runs per 7-day window
• Error budget: <=0.1% critical failures per 7-day window
• Alert triggers:
• credential validation failures exceed baseline
• schema or contract regressions persist for two consecutive runs
• critical posture or rollback events exceed tolerance
• Primary outcome metric: report freshness
• Secondary metrics: metric correctness, delivery timeliness
• Review cadence: daily