srednoff888-art AI Evals Regression Suite
Use this skill for AI Engineering tasks focused on LLM evals, golden datasets, structured outputs, drift detection, regression gates.
Workflow
- Clarify the user outcome, constraints, current stack, and definition of done.
- Inspect local source, artifacts, analytics exports, logs, designs, or docs before changing anything.
- Check official documentation when APIs, SDKs, policies, search behavior, accessibility rules, or production behavior may have changed.
- Compare proven open-source patterns for non-trivial choices and adapt ideas without copying incompatible code.
- Produce the smallest production-ready change or recommendation that satisfies the request.
- Validate with relevant evidence: tests, lint, typecheck, build, screenshots, crawl output, logs, analytics, or manual scenario.
- Report commands, files, findings, risks, and next steps clearly.
Focus Checklist
- Map inputs, owners, dependencies, and constraints.
- Prefer existing project conventions, design systems, and platform primitives.
- Include failure states, privacy/security implications, performance impact, and rollback where relevant.
- Separate facts from assumptions and mark any unverified claims.
- Keep recommendations actionable and prioritized by impact and risk.
Guardrails
- Do not judge AI quality only from a single hand-picked example.
- Do not perform destructive, paid, production, publishing, account-changing, or externally visible actions without explicit user confirmation.
- Do not expose secrets, private keys, tokens, cookies, private analytics, personal data, or confidential business data.
- If validation is impossible, state why and provide a concrete manual verification path.