all publishers

arize-ai

@arize-ai source repo

50 published skills

  1. ▌
    Check Models 2 · arize-ai bundle
    Find and update out-of-date OpenAI and Anthropic model references (in docs, MDX, notebooks, and code) to the latest size-equivalent models, and apply the code changes each new generation requires (e.g. max_tokens → max_completion_tokens for GPT-5). Use when asked to "check the models", "update model versions", "are these models current", "migrate the models in the docs/tutorials", or before publishing content that names a model.
    8 repo stars
  2. ▌
    Check Models · arize-ai bundle
    Find and update out-of-date OpenAI and Anthropic model references (in docs, MDX, notebooks, and code) to the latest size-equivalent models, and apply the code changes each new generation requires (e.g. max_tokens → max_completion_tokens for GPT-5). Use when asked to "check the models", "update model versions", "are these models current", "migrate the models in the docs/tutorials", or before publishing content that names a model.
    8 repo stars
  3. ▌
    Phoenix Evals · arize-ai bundle
    Build and run evaluators for AI/LLM applications using Phoenix.
    8 repo stars
  4. ▌
    Phoenix Tracing · arize-ai bundle
    OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.
    8 repo stars
  5. ▌
    Debug Setup · arize-ai
    Use when validating Claude Code telemetry setup for this collector, especially to confirm Claude emits OTLP data to a local file before testing the configured upstream destination.
    8 repo stars
  6. ▌
    Debug Live Collector · arize-ai
    Use when testing an already-running Claude collector by sending a live marker trace to its configured OTLP endpoint without starting a second collector.
    8 repo stars
  7. ▌
    Context Graph Apply · arize-ai
    INVOKE THIS SKILL when translating a context-graph-mining report into an experiment variant for the procurement-agent. Reads the report's proposed diffs and writes a variant config bundle to `experiments/variants/<id>/` (manifest.yaml + system_prompt.txt + optional vendors.json + departments.json). Does NOT modify procurement-agent source — runtime parameterization means the variant is purely additive config. Use after `context-graph-mining` produces a report; output feeds `procurement-experiment run-experiment`.
    8 repo stars
  8. ▌
    Context Graph Mining · arize-ai
    INVOKE THIS SKILL when mining the procurement-agent Arize project for patterns in agent decisions vs human overrides, building a context graph, and proposing updates to procurement-agent rules. Use after a batch of reviews has been captured (process.run + override.run spans with annotations) to surface (a) where the agent's recommendations systematically diverge from reviewer overrides, (b) tribal knowledge embedded in reviewer reasoning that should become explicit rules, and (c) concrete code-change proposals against the procurement-agent codebase.
    8 repo stars
  9. ▌
    Arize Datasets · arize-ai
    Manage datasets in Arize AI using the ax CLI. Use when users want to list datasets, get dataset details, create new datasets, delete datasets, export dataset data, or work with dataset examples. Triggers on "list datasets", "create dataset", "ax datasets", "export dataset", "delete dataset", or any request about managing Arize datasets via CLI.
    8 repo stars
  10. ▌
    Arize Projects · arize-ai
    Manage projects in Arize AI using the ax CLI. Use when users want to list projects, get project details, create new projects, delete projects, or organize work within Arize spaces. Triggers on "list projects", "create project", "ax projects", "delete project", or any request about managing Arize projects via CLI.
    8 repo stars
  11. ▌
    Setup Claude Code Tracing · arize-ai
    Set up and configure Arize tracing for Claude Code sessions or Agent SDK applications. Use when users want to set up tracing, configure Arize AX or Phoenix, create a new Arize project, get an API key, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up tracing", "configure Arize", "configure Phoenix", "enable tracing", "setup-claude-code-tracing", "create Arize project", "get Arize API key", "agent sdk tracing", or any request about connecting Claude Code or the Agent SDK to Arize or Phoenix for observability.
    8 repo stars
  12. ▌
    Code Review · arize-ai
    Use before opening a pull request to this repo. Reviews the working diff for correctness bugs and repo conventions, runs the test/lint gates, and can post findings directly on the PR. Triggers on "review my changes", "review this PR", "pre-PR review".
    8 repo stars
  13. ▌
    Phoenix CLI · arize-ai bundle
    Debug LLM applications using the Phoenix CLI. Fetch traces, analyze errors, structure trace review with open coding and axial coding, inspect datasets, review experiments, query annotation configs, and use the GraphQL API. Use whenever the user is analyzing traces or spans, investigating LLM/agent failures, deciding what to do after instrumenting an app, building failure taxonomies, choosing what evals to write, or asking "what's going wrong", "what kinds of mistakes", or "where do I focus" — even without naming a technique.
    8 repo stars
  14. ▌
    Manage Omp Tracing · arize-ai
    Set up and configure Arize tracing for Oh My Pi (omp) terminal coding sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Oh My Pi, enable/disable omp tracing, or troubleshoot tracing issues. Triggers on "set up omp tracing", "configure Arize for Oh My Pi", "configure Phoenix for omp", "enable omp tracing", "setup-omp-tracing", or any request about connecting omp / Oh My Pi to Arize or Phoenix for observability.
    8 repo stars
  15. ▌
    Manage Kiro Tracing · arize-ai
    Set up and configure Arize tracing for Kiro CLI sessions. Use when users want to set up Kiro tracing, configure Arize AX or Phoenix for Kiro, enable/disable tracing, choose or set a default traced agent, or troubleshoot Kiro tracing issues. Triggers on "set up kiro tracing", "configure Arize for Kiro", "configure Phoenix for Kiro", "enable kiro tracing", "setup-kiro-tracing", "kiro agent tracing", or any request about connecting Kiro CLI to Arize or Phoenix for observability.
    8 repo stars
  16. ▌
    Manage Codex Tracing · arize-ai
    Set up and configure Arize tracing for OpenAI Codex CLI sessions. Use when users want to set up Codex tracing, configure Arize AX or Phoenix for Codex, enable/disable tracing, or troubleshoot Codex tracing issues. Triggers on "set up codex tracing", "configure Arize for Codex", "configure Phoenix for Codex", "enable codex tracing", "setup-codex-tracing", or any request about connecting Codex to Arize or Phoenix for observability.
    8 repo stars
  17. ▌
    Manage Cursor Tracing · arize-ai
    Set up and configure Arize tracing for Cursor IDE sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Cursor, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up cursor tracing", "configure Arize for Cursor", "configure Phoenix for Cursor", "enable cursor tracing", "setup-cursor-tracing", or any request about connecting Cursor to Arize or Phoenix for observability.
    8 repo stars
  18. ▌
    Manage Gemini Tracing · arize-ai
    Set up and configure Arize tracing for Gemini CLI sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Gemini, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up gemini tracing", "configure Arize for Gemini", "configure Phoenix for Gemini", "enable gemini tracing", "setup-gemini-tracing", or any request about connecting Gemini CLI to Arize or Phoenix for observability.
    8 repo stars
  19. ▌
    Manage Copilot Tracing · arize-ai
    Set up and configure Arize tracing for GitHub Copilot sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Copilot, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up copilot tracing", "configure Arize for Copilot", "configure Phoenix for Copilot", "enable copilot tracing", "setup-copilot-tracing", or any request about connecting GitHub Copilot to Arize or Phoenix for observability.
    8 repo stars
  20. ▌
    Manage Opencode Tracing · arize-ai
    Set up and configure Arize tracing for opencode terminal coding sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for opencode, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up opencode tracing", "configure Arize for opencode", "configure Phoenix for opencode", "enable opencode tracing", "setup-opencode-tracing", or any request about connecting opencode to Arize or Phoenix for observability.
    8 repo stars
  21. ▌
    Manage Antigravity Tracing · arize-ai
    Set up and configure Arize tracing for Google Antigravity CLI/IDE sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Antigravity, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up antigravity tracing", "configure Arize for Antigravity", "configure Phoenix for Antigravity", "enable antigravity tracing", "setup-antigravity-tracing", or any request about connecting the Antigravity CLI/IDE to Arize or Phoenix for observability.
    8 repo stars
  22. ▌
    Manage Claude Code Tracing · arize-ai
    Set up and configure Arize tracing for Claude Code sessions or Agent SDK applications. Use when users want to set up tracing, configure Arize AX or Phoenix, create a new Arize project, get an API key, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up tracing", "configure Arize", "configure Phoenix", "enable tracing", "setup-claude-code-tracing", "create Arize project", "get Arize API key", "agent sdk tracing", or any request about connecting Claude Code or the Agent SDK to Arize or Phoenix for observability.
    8 repo stars
  23. ▌
    Rosetta Aaaj · arize-ai
    Attach Agent-as-a-Judge (AaaJ / harness) evaluators to any Arize AX Wonder Toys project and trigger a one-time backfill. Reads scoring instructions from evals/aaaj/*.md and creates evaluators + tasks via GraphQL (REST and the ax CLI cannot create harness evals). Use when the user asks to add AaaJ, agent-as-a-judge, harness evals, or to score AX traces without the UI. AX-only; not part of rosetta-test e2e. Default PROJECT_DIR is ax/langchain-py; any ax/<framework> with .env.local works.
    8 repo stars
  24. ▌
    Rosetta Test · arize-ai
    End-to-end test a Rosetta Stone framework × platform combination. Provisions a fresh isolated project on Arize AX or Phoenix, generates 25 synthetic traces, runs the 6 evals, verifies every trace was evaluated, then deletes the project. Trigger when the user asks to "test the <framework> <platform> project", "run e2e on <framework> <platform>", "verify <framework> works on <platform>", or any similar phrasing. Frameworks and platforms are discovered from the directory layout — no hardcoded list, so this works for any new framework added under ax/ or phoenix/.
    8 repo stars
  25. ▌
    Rosetta Test Evals · arize-ai
    Run the 6 Wonder Toys evals against a Rosetta Stone test project. Phoenix path runs the built-in `npm run evals` script. AX path ensures the stable space-level rosetta-e2e-* evaluators exist (creating only missing ones from the templates in evals/README.md), then creates and triggers a per-run eval task scoped to the project. Part of the rosetta-test e2e flow; can also be invoked standalone.
    8 repo stars
  26. ▌
    Rosetta Test Setup · arize-ai
    Provision a fresh Rosetta Stone test project on Arize AX or Phoenix. Validates the target framework directory and credentials, mints a unique project name, writes a sibling .env.test-local overlay so the real .env.local is untouched, and pre-creates the AX project (Phoenix auto-creates on first trace). Part of the rosetta-test e2e flow; can also be invoked standalone.
    8 repo stars
  27. ▌
    Rosetta Test Traces · arize-ai
    Run the 25 synthetic Wonder Toys requests against a Rosetta Stone framework to generate traces in an isolated Arize AX or Phoenix project. Loads the .env.test-local overlay so the project name override takes effect without touching .env.local. Part of the rosetta-test e2e flow; can also be invoked standalone if setup already ran.
    8 repo stars
  28. ▌
    Rosetta Test Verify · arize-ai
    Verify a Rosetta Stone test project has 25 traces and every trace has every expected eval annotation. Presence-only check — eval scores are not inspected. Reports a per-trace coverage matrix and pass/fail. Part of the rosetta-test e2e flow; can also be invoked standalone after evals have completed.
    8 repo stars
  29. ▌
    Rosetta Demo Capture · arize-ai
    Record a full Wonder Toys demo by running a canned 3-turn conversation (search dragons → buy plushie → ship), then opening Arize AX in Safari and screenshotting the session view plus every trace in it. Use when the user asks to "capture a demo", "record screenshots of an Arize session", "demo the agent flow", or any similar phrasing. macOS only — uses AppleScript and `screencapture`.
    8 repo stars
  30. ▌
    Rosetta Test Cleanup · arize-ai
    Tear down a Rosetta Stone test project — deletes the platform project on Arize AX or Phoenix, removes the .env.test-local overlay, and kills any leftover dev server / ChromaDB processes started by the run. Idempotent and safe to call multiple times. Use the --keep flag at the orchestrator level to skip this entirely. Part of the rosetta-test e2e flow; can also be invoked standalone to clean up after a crashed run.
    8 repo stars
  31. ▌
    Rosetta Add Framework · arize-ai
    Add a new agent framework to the Rosetta Stone repo — researches the framework, builds all three observability tiers (no-observability, phoenix, ax), tests each, runs a Playwright smoke against the UI, updates README + TODO, and raises a PR. Trigger when the user asks to "add the <framework> framework", "implement <framework>", "wire up <framework>", or similar. The framework must be one of the Arize-supported agent frameworks (see TODO below).
    8 repo stars
  32. ▌
    Rosetta Pr Screenshots · arize-ai
    Capture AX trace UI, Phoenix trace UI, and Wonder Toys app UI screenshots for a framework's PR, then upload as GitHub release assets and embed in the PR body. Called automatically by rosetta-add-framework-docs as part of new-framework PRs; can also be invoked standalone to retrofit existing PRs. Cross-platform — uses Playwright end-to-end.
    8 repo stars
  33. ▌
    Rosetta Add Framework Docs · arize-ai
    Finalise a newly-added framework — updates the README's supported-frameworks table, directory tree, and per-framework "what differs" section, marks off the framework in the orchestrator skill's embedded TODO, commits per tier, and raises a PR. Part of the rosetta-add-framework flow.
    8 repo stars
  34. ▌
    Rosetta Add Framework Discover · arize-ai
    Refresh the list of agent frameworks supported by Arize tracing and diff against what's already in the repo. Pulls live data from https://arize.com/docs/llms.txt and produces a clean to-do list. Part of the rosetta-add-framework flow; can also be invoked standalone to answer "what frameworks are left to add?"
    8 repo stars
  35. ▌
    Rosetta Add Framework Tier Test · arize-ai
    Test a freshly-built tier for a new framework — boots the backend, smoke-tests the chat endpoint, runs synthetic requests, and (for phoenix) runs the eval harness. Verifies traces land in the right project. Part of the rosetta-add-framework flow; can be invoked standalone after a build to validate.
    8 repo stars
  36. ▌
    Rosetta Add Framework Playwright · arize-ai bundle
    Run a public-flow Playwright smoke test against a freshly-built framework tier's Next.js frontend. Covers home page rendering and product browsing. The Playwright project (package.json, config, tests) lives inside this skill directory and is checked into the repo. Part of the rosetta-add-framework flow.
    8 repo stars
  37. ▌
    Rosetta Add Framework Tier Build · arize-ai
    Build a single tier (no-observability, phoenix, or ax) for a new framework. Clones the closest existing tier, swaps in framework-specific agent.py / tools.py / requirements.txt, and (for observability tiers) adds tracing.py + main.py wiring + eval-harness scripts. Part of the rosetta-add-framework flow; can be invoked standalone to rebuild a single tier from scratch.
    8 repo stars
  38. ▌
    Arize Link · arize-ai bundle
    Generates deep links to the Arize UI for projects, traces, spans, sessions, datasets, labeling queues, evaluators, and annotation configs. Discovers organization and project IDs with the ax CLI and produces clickable URLs for sharing Arize resources with team members. Use when the user wants to link to or open a project, trace, span, session, dataset, evaluator, or annotation config in the Arize UI.
    8 repo stars
  39. ▌
    Arize Admin · arize-ai bundle
    Manages Arize users, organizations, spaces, projects, roles, role bindings, resource restrictions, and API keys via the ax CLI. Use for enterprise admin workflows: inviting and offboarding users, onboarding new teams, creating custom roles for SAML/SSO mappings, assigning roles to users, restricting project-level access, and managing service keys for multi-tenant architectures. Covers ax users, ax organizations, ax spaces, ax projects, ax roles, ax role-bindings, and ax api-keys.
    8 repo stars
  40. ▌
    Arize Trace · arize-ai bundle
    Downloads, exports, and inspects existing Arize traces and spans to understand what an LLM app is doing or debug runtime issues. Covers exporting traces by ID, spans by ID, sessions by ID, and root-cause investigation using the ax CLI. Use when the user wants to look at existing trace data, see what their LLM app is doing, export traces, download spans, investigate errors, or analyze behavior regressions.
    8 repo stars
  41. ▌
    Arize Dataset · arize-ai bundle
    Creates, manages, and queries Arize datasets and examples. Covers dataset CRUD, appending examples, exporting data, and file-based dataset creation using the ax CLI. Use when the user needs test data, evaluation examples, or mentions create dataset, list datasets, export dataset, append examples, dataset version, golden dataset, or test set.
    8 repo stars
  42. ▌
    Arize Prompts · arize-ai bundle
    INVOKE THIS SKILL for Arize Prompt Hub and `ax prompts` workflows: author or import templates and save (Workflows A–B), label/promote (C), or list/get/edit/delete/duplicate (D). Use when the user mentions ax prompts, Prompt Hub, creating/editing/saving a prompt, `{variable}` placeholders, or production/staging labels. For improving prompt text using traces or eval scores, use arize-prompt-optimization. For running experiments, use arize-experiment.
    8 repo stars
  43. ▌
    Arize Evaluator · arize-ai bundle
    Handles LLM-as-judge and code evaluator workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, code evaluator, hallucination, faithfulness, correctness, relevance, run eval, score spans, score experiment, trigger-run, column mapping, continuous monitoring, or improve evaluator prompt.
    8 repo stars
  44. ▌
    Arize Annotation · arize-ai bundle
    Creates and manages annotation configs (categorical, continuous, freeform label schemas) and annotation queues (human review workflows) on Arize. Applies human annotations to project spans via the Python SDK. Use when the user mentions annotation config, annotation queue, label schema, human feedback, bulk annotate spans, update_annotations, labeling queue, annotate record, or human review.
    8 repo stars
  45. ▌
    Arize Experiment · arize-ai bundle
    Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance. Covers experiment CRUD, exporting runs, comparing results, and evaluation workflows using the ax CLI. Use when the user mentions create experiment, run experiment, compare models, model performance, evaluate AI, experiment results, benchmark, A/B test models, or measure accuracy.
    8 repo stars
  46. ▌
    Arize Span Routing · arize-ai bundle
    Use when one Python service must send each agent's, tenant's, team's, or request's spans to its correct Arize space and project using application metadata. Covers dynamic OpenTelemetry routing for custom agent builders and multi-tenant applications, including register_with_routing, set_routing_context, multi-space tracing, and custom span routing.
    8 repo stars
  47. ▌
    Arize Instrumentation · arize-ai bundle
    Adds Arize AX tracing to an LLM application for the first time. Detects the stack, routes to the single matching integration doc, wires auto-instrumentation after user confirmation, and verifies traces land. Use when the user wants to instrument their app, add tracing from scratch, set up LLM observability, integrate OpenTelemetry or openinference, or get started with Arize tracing.
    8 repo stars
  48. ▌
    Arize Prompt Optimization · arize-ai bundle
    Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement.
    8 repo stars
  49. ▌
    Arize Instrumentation Health · arize-ai bundle
    Audits instrumentation health of existing Arize traces. Runs deterministic checks over a bounded span sample (orphaned/uncategorized/duplicate spans, flat structure, blank root I/O, unset status, missing token counts or children) and returns a ranked report. Use when the user asks why traces look empty/flat/broken, wants to verify instrumentation is healthy, find instrumentation issues, or why evals or token/cost dashboards show n/a or zero. To debug app behavior or errors, use arize-trace.
    8 repo stars
  50. ▌
    Arize AI Provider Integration · arize-ai bundle
    Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and other Arize features. Supports any LLM provider (e.g. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM). Use when the user mentions AI integration, LLM provider credentials, create integration, list integrations, update credentials, delete integration, or connecting an LLM provider to Arize.
    8 repo stars