wandb
- 39 skills
- 0 followers
- 37 repo stars
- last month last updated
- ▌ Wandb Primary 2 · wandb bundleComprehensive primary skill for agents working with Weights & Biases. Covers both the W&B SDK (training runs, metrics, artifacts, sweeps) and the Weave SDK (GenAI traces, evaluations, scorers). Includes helper libraries, gotcha tables, and data analysis patterns. Use this skill whenever the user asks about W&B runs, Weave traces, evaluations, training metrics, loss curves, model comparisons, or any Weights & Biases data — even if they don't say "W&B" explicitly.
- ▌ Convert Datadog Dashboard · wandb bundleConvert a DataDog dashboard JSON into a Grafana dashboard JSON that fits the operator's telemetry stack (VictoriaMetrics datasources, wandb-<component> UID, no templating). Trigger when the user provides DataDog dashboard JSON (inline or as a file path) and asks to port, convert, or recreate it as a Grafana dashboard, or when the user asks to add a new dashboard under deploy/telemetry/dashboards/ from a DataDog source.
- ▌ Weave Integration · wandb bundleComprehensive skill for adding W&B Weave to existing applications. Covers trace-first instrumentation, evaluation only after trace verification, documentation-first implementation, CLI-based Weave data access, and validation workflows. Activate this skill only when the user explicitly mentions the skill by name, such as `weave-integration`, `@skills weave-integration`, or another direct skill reference. Do not auto-trigger from generic Weave or W&B questions.
- ▌ Horangi Fails · wandbDeep-dive error pattern analysis for a single (model, benchmark) pair using Weave traces. Surfaces how/why the model is getting answers wrong — answer bias, format violations, language mixing, and 3-5 representative failure samples. Invoke when the user asks to analyze wrong answers / failure patterns for a specific benchmark (e.g. "analyze errors in <bench>", "<model>의 <benchmark> 오답 패턴 분석", "틀린 문제 경향"). Commonly invoked as a follow-up to `horangi-analyze` when that skill flags a weak category.
- ▌ Horangi Analyze · wandbAnalyze one or more Horangi model configs against (1) HuggingFace model card claims, (2) the W&B leaderboard rankings, and (3) category-level peer comparison. Invoke when the user asks to analyze/compare model performance for a config (e.g. "analyze <config>", "compare X and Y", "이 모델 성능 분석해줘").
- ▌ Build Verdict Dataset · wandbGuides coding agents through building the verdict_quality_dataset from W&B Weave research_annotation evidence — querying annotated research_run calls, mapping human QualitySelector verdicts to gold labels, refining row inputs per the rubric, and publishing a new versioned Weave Dataset. Use when the user asks to (re)generate the verdict dataset from annotations, seed a new eval dataset version, or rebuild verdict_quality_dataset.
- ▌ Annotation Improvement · wandbGuides coding agents through Discovery Forge prompt improvement after selecting the `wandb-primary` skill. Use `wandb-primary` to fetch W&B Weave research_run traces, human annotations, runnable feedback, and evaluations, then use this workflow when improving researcher.md from annotation queues, reviewed research traces, or human feedback.
- ▌ Offline Eval Improvement · wandbGuides coding agents through Discovery Forge prompt improvement from a specified Weave offline evaluation baseline. Use `wandb-primary` to fetch evaluation results, failed eval rows, dataset refs, prompt refs, and scorer evidence, then use this workflow when improving researcher.md from failed evaluation rows, comparing eval runs, or iterating on a fixed dataset.
- ▌ Weave Config · wandbThis skill should be used when the user wants to "configure weave", "set weave project", "change weave project", "set wandb api key", "update weave settings", "show weave config", "change weave configuration", "restart the weave daemon", "apply weave config changes", "restart weave to pick up changes", or needs to read or update any Weave Claude Code plugin settings.
- ▌ Weave Status · wandbThis skill should be used when the user wants to "check weave status", "verify the weave plugin is running", "see if weave is set up correctly", "check weave configuration", "is weave working", "weave is running an older config", "the daemon is on an old config", or needs to diagnose why Claude Code sessions are not appearing in Weave.
- ▌ Weave Install · wandbThis skill should be used when the user wants to "install the weave plugin", "set up weave", "install weave-claude-code", "configure weave for the first time", "get started with weave tracing", or needs to complete the initial setup of the Weave Claude Code plugin including dependency installation and project configuration.
- ▌ Weave Instrument · wandb bundleAdd Weave (Weights & Biases) observability to an LLM or agent codebase. This covers calling `weave.init()`, setting up authentication, and choosing between OTEL auto-instrumentation and the explicit Session SDK agent-logging APIs (Turn, LLM, Tool, SubAgent), based on the libraries the code already uses. Works for Python and TypeScript/Node. Use this whenever the user wants to instrument, trace, or add observability, logging, or monitoring to an agent, chatbot, RAG pipeline, or LLM app. This includes phrasings like "log my agent to weave", "add agent tracing", "get my agent into the Weave Agents tab", "instrument this with the weave session sdk", "trace my tool calls", or "set up weave logging", even when the user does not name a specific API.
- ▌ Bump Version · wandbBump the Weave Python SDK version for release. Use when preparing a new release.
- ▌ Publish Pypi · wandbBuild and publish the Weave Python SDK to PyPI. Use when releasing a new version.
- ▌ Ship · wandbBranch, commit, and/or open a PR for the current changes — driven by AskUserQuestion. Use when the user wants to commit their work, create a branch, open a pull request (draft or ready), refine an existing PR's description, set the release label, or "ship" / "wrap up" what they've been working on. Produces conventional commits and a PR that fits the repo template and passes the PR Checks gate.
- ▌ Sync Operator · wandbSync wsm to be e2e compatible with a target github.com/wandb/operator ref — always invokes /audit-operator-diff first, then applies the user-approved fix list, updates docs, and smoke-tests on Kind. Triggers on phrases like "sync wsm with operator", "bump operator dep", "update for new operator release", "match operator changes". v2 surface only by default; v1 only on explicit user request. Requires a local clone of the operator repo and OrbStack running.
- ▌ Audit Operator Diff · wandbAudit changes between two github.com/wandb/operator refs and produce a categorized impact report on wsm. Use independently for code review / release planning, or as Step 1 of /sync-operator. Triggers on phrases like "audit operator diff", "what changed in operator", "show me operator impact on wsm", "operator compatibility audit". Does NOT make code changes — produces a fix list at .claude/audit-report.md that the user (or /sync-operator) consumes.
- ▌
- ▌ Bootstrap Target · wandb bundleCreate or improve a Senpai target repository's program.md. Use this skill whenever the user wants to point Senpai at a fresh ML or research target repository, define the research objective, primary metric, benchmark contract, allowed edit boundaries, W&B reporting contract, or prepare a repo for autonomous advisor/student experiment loops.
- ▌ Git Research Log · wandb bundleDeveloper reference for Senpai's retired direct-GitHub experiment logging workflow. Use only when auditing or migrating legacy research tracks.
- ▌ List Experiments · wandbOperator-side inventory of experiment PRs for auditing a Senpai research track. It outputs merged winners, a compact results table, and full details for deep dives.
- ▌ Experiment Report · wandb bundleCreate consistent nn_cfd W&B experiment comparison reports from selected run IDs using wandb and wandb_workspaces. Use this whenever the user asks to report experiment results, create a W&B Report, compare CFD training/eval runs, document baseline-vs-variant outcomes, reproduce the project experiment report template, or publish a leaderboard-style report for milieu/nn_cfd.
- ▌ Exa Search · wandb bundleSearch the general web or scholarly publications through Exa. Use for current public information, official documentation, source code, release notes, papers, preprints, journals, or literature research.
- ▌ Analyze Experiments · wandb bundleOperator-side analysis of historical ML experiment PRs in Senpai research tracks. Use this skill whenever the user asks to: analyze experiments, categorize PRs, bucket experiments, summarize what's been tried, understand experiment history, review merged vs closed results, or asks "what experiments have we run / worked / failed". Also triggers for: "pull the latest experiments", "what's been tried so far", "category breakdown of PRs", "which experiments succeeded", "noam track analysis". When a branch name is mentioned (e.g. "noam branch", "on the noam branch"), pass it as the base branch to scope the fetch to just those PRs.
- ▌ Grilling Autoresearch · wandb bundleCoach the user through a rigorous, repository-grounded interview to create or improve program.md. Use when a target repository lacks program.md, its goals, metrics, data, or guardrails are unclear, or the user asks to be grilled about an autoresearch setup.
- ▌ Senpai Tool Telemetry · wandb bundleAudit recent tool use across local Senpai OpenHands root and recursively delegated conversations without exposing tool arguments. Use for provider comparisons, tool-loop diagnosis, status-poll analysis, error rates, and latency reports.
- ▌ Plot Experiment Charts · wandbOperator-side guide for generating a target-specific training curve chart for a legacy experiment PR. Use when auditing or presenting historical runs.
- ▌ Assign Experiment · wandbCreate a typed assignment branch and draft PR for one student. Use when the advisor has a concrete hypothesis and the student has no open `status:wip` or `status:review` assignment.
- ▌ Review Experiment · wandbReview one terminal experiment and choose the next step: merge a reproducible winner, close a useful negative or dead end, or request missing evidence.
- ▌ Check Human Issues · wandbCheck and respond to GitHub Issues from the human researcher team. Use this skill whenever you need to handle a human_issue event, respond to human issues, or check team communications. Also triggers for: "any human messages?", "check issues", "respond to humans".
- ▌ Delegate Subagents · wandb bundleChoose, configure, launch, and collect bounded subagents for research and engineering decisions. Root agents and delegation-capable subagents should read this before delegating: every task requires an explicit model tier, and high-leverage work such as research ideation, round planning, plateau pivots, large research reviews, hard optimization, disputed evidence, and expensive experiment portfolios requires frontier judgment.
- ▌ Senpai Status Check · wandbProduce a fresh, read-only status report for a Senpai research track from GitHub, W&B, and the current agent's local controller state.
- ▌ Maintain Research State · wandbMaintain and publish the advisor-owned current research state, durable research-ideas syntheses, and dataset analysis. Use when evidence or human guidance changes the research focus or direction, durable research ideas should be recorded, or dataset understanding changes.
- ▌ Submit Experiment Results · wandbPost material interim assignment updates, then commit and submit a terminal experiment result through the typed Senpai GitHub tools.
- ▌
- ▌ Marimo Example Notebook · wandbCreate or refactor a high-quality marimo example notebook for the wandb/examples repo. Use whenever adding a new example under examples/marimo/, converting a Jupyter example to marimo, or reviewing/refactoring an existing marimo example. Encodes this repo's structure conventions and W&B integration patterns.
- ▌ Wandb Primary · wandb bundlePrimary W&B skill for broad or mixed Weights & Biases work: project overviews, W&B runs and artifacts, Weave traces and evaluations, Reports, and Launch workflows. Use when the task spans multiple W&B surfaces or the user asks generally what is happening in a W&B project.
- ▌ Wandb Eval Tables · wandb bundleConvert W&B Table artifacts into non-destructive EvalTable previews with scan-first planning, typed input/output/score columns, bounded batches, verification, and safe removal. Use when a coding agent needs to create, inspect, compare, verify, or remove W&B EvalTable previews.
- ▌ Wandb Autoresearch · wandb bundleRun bounded, evidence-driven training research through W&B Launch: assess project readiness, establish launchable code and queue capacity, smoke-test real jobs, execute serial trials, compare metrics, and persist resumable research state. Use when a coding agent is asked to autonomously test training hypotheses or tune a real W&B-tracked workload.