grafana AI SDK Parity Review
Use this skill to review a user-defined scope for problematic upstream parity
issues against the baseline declared in test/conformance/upstream.yaml.
The scope comes from the user request. It can be a PR, the current git diff, a branch range, a package, a directory, a provider, a bug report, or a feature area. If the user does not specify a scope, review the current git diff.
Workflow
- Read
test/conformance/upstream.yamlandtest/conformance/PARITY.md. - Resolve the review scope:
- PR: inspect the PR diff and relevant changed files.
- Current git diff: inspect staged and unstaged changes.
- Branch/range: inspect the diff against the requested base.
- Package/directory/provider: inspect the implementation and matching tests in that tree.
- Bug or feature: inspect the affected code path and any existing fixtures.
- Identify the parity-sensitive layer and capability from the coverage map: core ai-sdk, provider contract, provider implementation, frontend interop, or conformance harness.
- Read the matching upstream TypeScript implementation and tests for the
registered baseline when available. Prefer sources in this order:
- a local upstream checkout at the registered version or matching tag
- the conformance tool package installed from the registered baseline
- raw GitHub source for the registered package version Record which source was used when reporting findings.
- Check conformance evidence appropriate for the layer:
expected.jsonlfor core stream/UI behavior,expected-requests.jsonlfor provider behavior,expected-object.jsonfor structured output, or a documented manual/gap entry. - For reported bugs or new upstream-visible behavior, check whether conformance-first TDD is possible. If so, prefer adding or updating the fixture before or alongside implementation.
- Classify each mismatch using the divergence handling table below.
- Report only problematic findings: missing features, behavioral deviations, possible bugs, missing or weak conformance coverage, and undocumented intentional deviations. Do not enumerate parity-preserving matches unless they explain why a suspected issue is not a problem.
- For broad scopes, include a compact matrix with the reviewed package/area, upstream files or tests checked, conformance artifacts checked, mismatch classification, action taken, and residual risk.
- For PR reviews, put accepted gaps, intentional deviations, and residual
parity risk in the PR description or review summary. Do not add broad
one-off findings to
test/conformance/PARITY.mdortest/conformance/upstream.yamlunless the user explicitly asks or the finding changes a long-lived project policy, coverage map, or baseline contract. - Include commands run and residual parity risk in the review summary.
Divergence Handling
| Classification | Default action |
|---|---|
| Implementation bug | Add or update the relevant fixture when practical, fix the Go implementation, and rerun focused checks. |
| Upstream behavior change | Trace the change to upstream source or tests, regenerate the matching conformance artifact, adapt Go behavior, and rerun parity checks. |
| Intentional Go deviation | Keep only with a rationale. Document in the PR description or review summary unless it changes a durable baseline contract. |
| Coverage gap | Add a fixture or test when cheap and relevant. Otherwise document the residual risk in the PR description or review summary. |
| Large design gap | Stop and ask before deferring when the gap affects public API shape, wire format, provider contracts, or broad harness behavior. |
Review Criteria
- Wire shape, chunk type names, JSON tags, ordering, and SSE framing must match upstream unless a documented deviation exists.
- Provider request bodies and behavior-affecting headers must match upstream request snapshots unless a documented normalization rule applies.
- Go implementation can use Go idioms, but upstream semantics must be preserved.
- New gaps must be fixed when practical. Accepted gaps and intentional
deviations must be visible in the PR description or review summary; reserve
test/conformance/PARITY.mdandtest/conformance/upstream.yamlupdates for durable coverage-map, baseline-contract, or user-requested documentation. - If conformance is not updated for a wire/provider-boundary change, the review must explain why existing coverage is sufficient.
- Provider implementation changes usually need
expected-requests.jsonlcoverage; core stream changes usually needexpected.jsonlcoverage. - If no upstream equivalent exists, state that and classify the compatibility impact.