cfregly

@cfregly source repo

35 skills
0 followers
6 hours ago last updated

▌
Agent Audit · cfregly bundle
Review an agent's tool inventory, tool calls, reasoning summaries, tool outputs, final answers, and value-bar proof using this repo's trace-review harness. Use when asked to audit, test, score, or review agent traces, Claude Messages API transcripts, tool schemas, trace suites, or agent audit bundles for tool-use quality, inter-tool reasoning, recovery behavior, final answer grounding, and whether the work is adversarially-confirmed to add value.

0
installs
▌
Human Docs Readability · cfregly bundle
Human-first editing standard for README.md files and user-facing Markdown docs. Use when Codex needs to rewrite, review, or gate public docs so humans see the value story first, links are easy to scan, modern GitHub Markdown components are used tastefully, and LLM or machine-readable audit details move into bottom collapsible sections while preserving adversarially-confirmed to add value evidence.

0
installs
▌
Your Skill Name · cfregly
Copy this directory to plugins/profile-and-optimize/skills/<your-skill-name>/ and rewrite this description to be third-person + include WHAT the skill does and WHEN (specific trigger phrases). Max 1024 chars. Do NOT leave the word "template" in the real skill's description.

0
installs
▌
Perf Baseline Diff · cfregly
Diff a current performance measurement against a registered baseline from the perf-baselines registry. Works for any measurement type (NCCL BW, MFU, step-time, latency, throughput, per-kernel times, structured heatmaps, nsys-rep profiles). Honors the operator-supplied tolerance, reports per-dimension deltas, and classifies the overall verdict as GREEN / YELLOW / RED. Pairs with perf-baseline-record. Triggers on "diff baseline", "diff against baseline", "perf regression check", "is this regression real", "compare to baseline", "perf-baseline-diff", "regression diff", or any combination of "diff / compare / regression / check" with "baseline / golden / reference / perf-of-record".

0
installs
▌
K8S Troubleshooting · cfregly
Expert Kubernetes troubleshooting assistant for diagnosing and resolving issues across the full stack - pods, control plane, nodes, networking, storage, and underlay infrastructure - in GPU cloud environments. Triggers on any report of a broken, degraded, or mysterious Kubernetes issue: pod crashes, OOMKills, scheduling failures, network problems, CRD errors, node NotReady, high latency, PVC issues, GPU/InfiniBand problems, workload hangs, or any cluster incident. Also triggers when the user pastes error messages, kubectl output, alert names, or incident-channel links and wants help understanding what's wrong. This skill works iteratively - it does NOT dump a wall of diagnostics all at once. It pauses after each step and asks the user how to proceed.

0
installs
▌
Evidence Bundle Init · cfregly
Scaffold a new evidence bundle directory ready for reproducibility-grade evidence capture: SOURCE.md (operator + cluster + git SHA + UTC timestamp), summary.md (verdict skeleton), commands/ (for the four-file .cmd/.stdout/.stderr/.exit tuple capture per shell command). Workload-agnostic. Works for any experiment family. Operator names family + run-id (or accepts default). Triggers on "new evidence bundle", "init evidence", "scaffold bundle", "start a new artifact bundle", "new run-id", "evidence-bundle-init", "set up a bundle", or any combination of "new / init / scaffold / create / start" with "evidence / bundle / artifact / run-id / experiment".

0
installs
▌
Inference Graph Diff · cfregly
Diff the compiled FX / Inductor graphs across two vLLM versions or two helm configs to see exactly which fused kernels / passes / partitions changed. Uses `torch._dynamo.explain` + `torch.compile`'s graph-dump hooks. Useful when you anticipate a model-graph-level change (e.g. swapping cudagraph_mode, enabling fuse_allreduce_rms, toggling pass_config.fuse_attn_quant). Triggers on "graph diff", "dynamo explain", "fx graph", "compile graph", "torch.compile diff", "compilation pass change", or any combination of "graph / fx / inductor / dynamo" with "diff / compare / explain / dump".

0
installs
▌
Inference Model Eval · cfregly
Drive lm-eval-harness quality evals (GPQA, MMLU-Pro) inside model pods plus optional ExternalEval (externally operated). Use to validate model quality before promoting to staging or prod, after vLLM / quantization / KV-cache changes, or to compare against published HuggingFace / paper baselines. Pair with inference-perf-bench (the perf-side counterpart) for full pre-promotion validation. Triggers on "lm-eval-harness", "GPQA", "MMLU-Pro", "ExternalEval", "model quality eval", "/run-model-eval", "run evals", "run gpqa", "run mmlu", "run external-eval", "run evals on the model", or any combination of "eval / quality / accuracy" with "inference / model / vllm".

0
installs
▌
Inference Perf Bench · cfregly
Canonical inference perf-bench skill (formal name. The colloquial alias is `ai-bench` - identical behaviour). Drives NVIDIA AIPerf + the replay-playback dataset against an in-cluster vLLM endpoint to measure TTFT, ITL, throughput, tok/s/user, request latency, and prefix cache hit rate. Iterative 9-phase workflow. Use when promoting a model to staging/prod, after vllmArgs / vLLM / KV-cache-dtype changes, or for A/B comparison across configs. Triggers on "perf-bench", "AIPerf", "Replay Playback", "throughput sweep", "TTFT P95", "concurrency sweep on inference", "/run-perf-bench", "benchmark throughput", "benchmark latency", "run aiperf", "performance test", or any combination of "perf / latency / throughput / tps / TTFT / ITL" with "inference / vllm / serverless / bench".

0
installs
▌
Inference Tune Sweep · cfregly
Search a bounded inference serving configuration matrix, compare every cell with a named baseline, and select a throughput or latency champion. Uses the checked-in campaign, evidence, baseline, aggregation, and report tools. Use when asked to tune batching, concurrency, KV cache, CUDA graph, or related serving settings for an existing deployment.

0
installs
▌
Perf Baseline Record · cfregly
Capture any performance measurement (NCCL bandwidth, MFU, step-time, latency, throughput, kernel-time, anything operator-defined) into a versioned baseline registry under experiments/artifacts/perf-baselines/. Workload-agnostic. Records full provenance (operator, cluster, cohort, git SHA, capture command, raw payload sha256) so future regressions can be diffed deterministically by the paired perf-baseline-diff skill. Triggers on "record baseline", "register baseline", "baseline this measurement", "save as baseline", "perf-baseline-record", "baseline perf-of-record", or any combination of "record / register / save / capture / store" with "baseline / golden / reference / perf-of-record".

0
installs
▌
Inference Aa Workload · cfregly bundle
Reproduce the Artificial Analysis (AA) language-model performance workload shapes against an OpenAI-compatible chat endpoint using NVIDIA AIPerf. Drives the three AA text shapes (1k input / >=1k answer, 10k / >=1.5k, 100k / >=2k) with temperature 0, top_p 1, and the vLLM-style min_tokens + ignore_eos "at least N answer tokens" guarantee. Two modes: synthetic (AIPerf generates the prompt at the token mean) and dataset-replay (a generated o200k_base-counted JSONL replayed identically). Ships a self-contained script and an `aa` perf_tune_report cell_run backend. Use when comparing a hosted inference endpoint to AA leaderboard numbers or reproducing AA's methodology. Triggers on "artificial analysis workload", "AA benchmark", "AA 1k/10k/100k shapes", "reproduce artificialanalysis.ai", "AA methodology", "aa-10k", "compare to AA leaderboard", or any combination of "artificial analysis / AA" with "workload / shape / benchmark / dataset".

0
installs
▌
Mirage Graph Coverage · cfregly bundle
Read-only coverage auditor for the mirage / MPK persistent-megakernel's GENERATED task graph: cross-references DECLARED tensors (all_tensors[...] in kernel_N.cu) against tensors CONSUMED by tasks (inputs/outputs base_ptr in task_graph_N.json) and flags any tensor declared but never consumed as an input - the "feature defined in code but never wired into the task graph" bug class (e.g. RoPE cos/sin tables allocated but consumed by 0 tasks => position-blind attention => incoherent / degenerate-repetition output). A numerics / relL2 harness silently PASSES on this class, so run THIS first when a megakernel compiles + exits 0 but emits incoherent output. Distinct from inference-graph-diff (vLLM FX / Inductor graphs). Triggers on "graph coverage", "task graph audit", "megakernel wiring", "declared but unconsumed tensor", "rope not wired", "missing task", or "megakernel / mirage / MPK / task-graph" with "coverage / wiring / unconsumed / unwired / audit".

0
installs
▌
Inference Value Ledger · cfregly
Render the leadership value-prop ledger for the inference effort: the deployable wins vs the FlashInfer-TRTLLM + best-tuned-vLLM 0.21/0.22 baseline, grouped DONE / IN-PROGRESS / NOT-DONE / CLOSED-NEGATIVE, each row data-backed by a perf-lake campaign (live sol_rigor + verdict tier), plus the ranked GRIND FRONTIER of next levers (the always-be-grinding performance ratchet). Joins the curated perf-tune-report/configs/value-findings.yaml registry with the live campaigns via `perftunereport value_view`. Flags any finding whose backing campaign is missing or ungrounded. Use when you need to show value / report to leadership / answer "what have we uncovered that we can deploy, revalidate, or pursue". Triggers on "value ledger", "value prop", "show value", "show the value prop", "leadership view", "what wins do we have", "vs flashinfer", "value-findings", "what can we deploy", or any combination of "value / wins / findings / leadership / report" with "inference / vllm / flashinfer / perf".

0
installs
▌
Zymtrace Anchored Query · cfregly
Reusable wrapper for the knowledge-base-first SQL pattern, adapted to the zymtrace ClickHouse profiling backend. Operator names the metric / question / time range. Skill anchors the `zymtrace_profiling.events` schema first (DESCRIBE + label-value + cardinality probes), derives the safe narrow SQL, runs it via `kubectl port-forward` + `curl -X POST`, and saves the raw payload to a provenance-bearing bundle per the perf-lake-contract. Workload-agnostic. The ClickHouse cousin of `prometheus-anchored-query`. Triggers on "zymtrace anchored query", "clickhouse anchored query", "zymtrace query", "save zymtrace payload", "anchored zymtrace", "anchored clickhouse", or any combination of "zymtrace / clickhouse / profile" with "anchored / safe / provenance / query / saved-payload".

0
installs
▌
Inference Dcgm Correlate · cfregly
Correlate DCGM Prometheus byte-traffic counters with an inference-perf-bench sweep window to compute byte-grounded workload-level Speed-of-Light. The third tier of the SoL rigor hierarchy (after zymtrace sample-share and ncu per-kernel arithmetic intensity). Reads the sweep window from `inference_perfbench_v1.json.bench`, queries Prometheus via the Prometheus MCP for the DCGM PROF group (`DRAM_ACTIVE`, `NVLINK_TX/RX_BYTES`, `PIPE_TENSOR_ACTIVE`, `PIPE_FP16_ACTIVE`), falls back to `DCGM_FI_DEV_*` counter-tier metrics when PROF is not exported, and writes `<cell>/dcgm_correlation.json` with per-resource %SoL = real byte traffic / (peak * duration * n_gpus). Triggers on "dcgm correlate", "dcgm sol", "byte-grounded sol", "workload %sol", "dram active over sweep", "nvlink bytes", "tensor pipe active", "real-vs-peak workload bandwidth", or combinations of "dcgm / prometheus" with "sweep / window / sol / workload / byte-traffic".

0
installs
▌
Inference Kernel Profile · cfregly
Capture per-kernel CUDA profile data from a live vLLM inference pod via an nsys debug sidecar (no production image rebuild). Outputs `.nsys-rep` + summary CSV + top-kernels table that joins with the zymtrace per-kernel breakdown and the inference-perf-bench bundle's `inference_perfbench_v1.json`. Triggers on "kernel profile", "nsys profile", "what's in the native bucket", "per-kernel breakdown", "ncu", "Nsight Systems", "kernel-level analysis", or any combination of "profile / capture / nsys / ncu" with "vllm / kimi / glm / deepseek".

0
installs
▌
Inference Model Optimize · cfregly
Coordinate an evidence-backed inference model optimization run from a user-supplied deployment or campaign configuration. Establishes readiness and a production-shaped baseline, profiles measured bottlenecks, runs a bounded tune sweep, checks quality, selects a champion, and renders a report. Use for a full model optimization pass. It does not scaffold an external deployment repository or depend on private helper scripts.

0
installs
▌
Analyze Zymtrace Workload · cfregly
Investigate a GPU or CPU workload through the zymtrace MCP. The MCP does most of the analysis. This skill enforces the cross-view discipline -- always pull the matching opposite-side flamegraph (CPU for GPU workloads, GPU for CPU workloads) with the same filter. Most bottlenecks hide on the side the customer didn't ask about. Triggers on "analyze my GPU workload", "where's the bottleneck in vllm", "investigate my training job", "find the hot kernel", "GPU isn't saturated", "investigate using flamegraph", "use zymtrace mcp to analyze", or any combination of "analyze / investigate / bottleneck / hot kernel" with "GPU / CPU / vllm / training / flamegraph / zymtrace".

0
installs
▌
Inference Capacity Sizing · cfregly
SLA-first GPU capacity sizing for a serving deployment: given a tokens-per-minute (TPM) target AND the interactivity SLA (output tokens/s/user), compute the pods and GPUs needed from a model's measured tok/s/user-vs-concurrency curve. Sizing MUST start from the tok/s/user SLA, not the TPM number alone: the SLA picks the concurrency, which sets per-pod throughput, which sets the GPU count. Also translates validated optimizations into delta-GPUs at the SLA-pinned concurrency. Backed by the standalone capacity_sizing.py tool. Use to answer "how many GPUs for N TPM", "size this for the customer", "pods for the SLA", or to sanity-check a sizing. Triggers on "capacity sizing", "how many GPUs", "how many pods", "tokens per minute sizing", "TPM target", "size for the customer", "interactivity SLA", "tokens per second per user", "GPU count for throughput", or any combination of "size / capacity / how-many" with "GPU / pod / TPM / SLA / tok-per-user / concurrency".

0
installs
▌
Inference Perf Synthesize · cfregly
Fuse the four profilers -- nsys (timeline + kernel durations), ncu (roofline / occupancy / arithmetic intensity), zymtrace (CPU+GPU flamegraph), and DCGM (byte-grounded %SoL) -- into ONE ranked, data-backed recommendation ledger for a vLLM deployment. Every recommendation (including concrete vLLM source/config changes) cites its backing artifact + sol_rigor tier, and each is staged as an A/B experiment so it is proven, not asserted. Emits a findings doc + perftunereport view + publishes to the perf-lake. The holistic-analysis phase of inference-model-optimize (Phase 2.5 + 8.5), usable standalone. Triggers on "synthesize the profiles", "what should I change to make this faster", "holistic perf analysis", "fuse nsys ncu zymtrace dcgm", "ranked perf recommendations", "suggest vllm changes from profiles", "explain the bottleneck with data", or any combination of "synthesize / fuse / recommend / holistic / explain" with "profile / nsys / ncu / zymtrace / dcgm / roofline / bottleneck / vllm".

0
installs
▌
Prometheus Anchored Query · cfregly
Reusable wrapper for the knowledge-base-first PromQL pattern. Operator names the metric / question / time range. Skill calls query_observability_knowledge_base first to confirm labels + cardinality, derives the safe narrow PromQL, runs it via the Prometheus MCP, and saves the raw payload to a provenance-bearing bundle per the perf-lake-contract. Workload-agnostic. Replaces the ad-hoc "run PromQL and hope" pattern with a disciplined anchor-then-query workflow. Triggers on "prometheus query", "promql query", "anchored query", "knowledge-base query", "query the Prometheus MCP", "what's the prometheus metric for", "save prometheus payload", or any combination of "promql / prometheus / metric / query" with "anchored / knowledge-base / safe / provenance".

0
installs
▌
Inference Perf Tune Report · cfregly
Build a multi-page benchmark report PDF (scatter panels + per-concurrency heatmap tables) from `vllm bench sweep serve_workload` and/or AIPerf outputs, in the GLM-5.1 reference layout (5x2 scatter facet by max_num_batched_tokens + 3x2 heatmap tables for C={8,16,32} x {tok/s/GPU, TTFT}. First-class failed/partial cells). Also renders the cross-engine (vLLM + SGLang) CHAMPION page 8: baseline vs top-X variants + the overlaid roofline + a RECOMMENDED-FOR-PRODUCTION pick via `champion_select`. Backed by the `perf_tune_report` CLI library. Triggers on "build a perf report", "render the atlas PDF", "perf-report campaign", "glm5p1-style report", "pareto report", "perftunereport smoke", "champion selection", "which variant to ship", "production champion", "baseline vs top variants", or any combination of "perf / atlas / report / champion" with "inference / vllm / sglang / aiperf / b200 / gb300 / nvfp4 / fp8".

0
installs
▌
Inference Spec Decode Tune · cfregly
Run a bounded comparison of speculative decoding draft training settings. Use measured serving acceptance and latency as the decision metrics, with training accuracy only as a triage signal. Use when one draft recipe already trains correctly and the operator can provide reproducible train and evaluation commands for each candidate.

0
installs
▌
Inference Workload Profile · cfregly bundle
Profile live inference traffic into a token/shape distribution artifact that drives profile-matched speculative-decoding draft training -- an analog of Fireworks FireOptimizer's "profile-driven customization" (the documented source of its higher draft hit-rate). Reads an OpenAI-style access JSONL, emits workload-profile.json (input/output length distributions, content-class mix, ISL/OSL bench shapes, and a spec-decode method recommendation), and hands off to inference-spec-decode-train via a hit-rate-matched corpus. The first phase of the adaptive spec-decode loop. Triggers on "profile my workload", "workload profile for spec-decode", "what draft should I train", "match the draft to my traffic", "adaptive speculative decoding", "fireoptimizer equivalent", "profile traffic for a draft model", or any combination of "profile / characterize / sample" with "workload / traffic / requests" and "spec-decode / draft / acceptance / hit-rate".

0
installs
▌
Inference Fleet Leaderboard · cfregly
Render cross-model fleet leaderboards from the local perf-report campaigns in ONE command (`perftunereport fleet_leaderboard`): a latency tier (aa-1k/10k/100k tok/s/user + TTFT + cost at c=1/c=10), a throughput tier (peak tok/s/GPU per (model,quant,TP) with latency + $/1M-tok), and a decision capstone (the perf Pareto frontier across decode-latency x first-token x throughput, with the perf-dominated set and a hard PERF != quality caveat). Auto-discovers every model's AA + roofline cells in campaigns/*/atlas.jsonl, so re-running refreshes as new campaigns publish. Use to answer "which model do I pick" / "rank the fleet on latency/throughput/cost" / "is model X perf-dominated". Triggers on "fleet leaderboard", "which model should I use", "cross-model comparison", "rank the fleet", "model selection guide", "pareto frontier of models", "perftunereport fleet_leaderboard", or any combination of "fleet / cross-model / which-model / leaderboard / pareto" with "latency / throughput / cost / pick / rank / compare".

0
installs
▌
Inference Known Good Config · cfregly
Capture + enforce per-model KNOWN-GOOD serving configs: the REQUIRED serve flags (boot-blockers / crash-at-high-c / deploy-correctness workarounds) plus the champion + the bug each flag avoids, in one queryable registry (perf-tune-report/configs/known-good-configs.yaml) so a hard-won workaround (e.g. Qwen3-Next's gdn_prefill_backend=triton on vLLM 0.22) is NEVER re-discovered the hard way. These are NOT well-known upstream defaults -- several are field-discovered workarounds where the upstream `auto` path is actively broken on the target hardware. `record` after a champion is found, `check` (fail-closed) before any deploy/ship. Keep the registry private -- never post an entry publicly. Triggers on "known-good config", "required flags for <model>", "did we capture the flag combo", "register the config", "check the deploy config", "config drift", "what flags does <model> need", or any combination of "known-good / required / champion config" with "record / check / register / capture / drift / flags / serve".

0
installs
▌
Inference Performance Hints · cfregly
Applies Jeff Dean and Sanjay Ghemawat's Performance Hints to GPU inference. Builds a back-of-the-envelope cost ledger, identifies the hot-path class, bounds each optimization with profile share, and routes the highest-value hypothesis to the repo's benchmark, profiling, tuning, quantization, or capacity skill. Use for "performance hints", "back-of-the-envelope perf", "where should I optimize", "flat profile", "hot path review", "estimate before benchmarking", or "make this inference path faster". Estimates remain DRAFT until production-shaped measurement confirms them.

0
installs
▌
Inference Spec Decode Train · cfregly
Train + validate a speculative-decoding draft head (EAGLE3 or DFlash) for an ARBITRARY target LLM, generalizing the GLM-5.1-only SpecForge recipe so any new model gets one. Parameterizes what that offline recipe hard-codes: target path, aux-hidden-states layers (derived [1, L/2-1, L-4] from config.json num_layers), chat template, draft-head config, and method. Wires the SpecForge prepare_hidden_states -> train_eagle3 / train_dflash -> convert-to-vLLM flow on Slurm, then gates on a measured acceptance-length A/B (dflash_vllm_eval.py + vLLM spec_decode_* counters) vs the standing config. This is the train-spec-decode phase of inference-model-optimize, usable standalone. Triggers on "train an EAGLE3 draft", "train a DFlash head", "build a speculative decoder for <model>", "spec-decode draft training", "draft model acceptance", "SpecForge for <model>", "generalize eagle3-train", or any combination of "train / build / validate" with "eagle3 / dflash / draft / speculative / spec-decode".

0
installs
▌
Inference Decode Step Budget · cfregly bundle
Measure the low-concurrency (c=1..c=8) decode hot-path of a live vLLM pod FAST and CORRECTLY: where each token's time actually goes (GPU-busy vs host-idle gap vs comm), whether the workload is kernel-bound / host-bound / comm-bound, and what the addressable lever is. Uses vLLM's native /start_profile + /stop_profile HTTP endpoints (enable once, then restart-free shell-in captures in seconds). Encodes the correctness gates that prevent contaminated or under-counted budgets (clean single-stream driver, GPU-busy must include CUDA-graph execution, reconcile against driver-measured TPOT, reject load-time/lull/truncated captures). Triggers on "decode step budget", "c=1 hot path", "where does decode time go", "TPOT breakdown", "is decode kernel-bound or host-bound", "latency-tier profile", "GPU idle during decode", "start_profile", "torch profiler vllm", or any combination of "decode / c=1 / latency / TPOT / ITL" with "profile / budget / breakdown / hot-path / where" on a vllm deploy.

0
installs
▌
Inference Kernel Ncu Profile · cfregly
Capture per-kernel CUDA hardware-counter data (occupancy, achieved warps active, regs/thread, smem/block, DRAM throughput, arithmetic intensity, branch divergence %, warp-stall reason %) from a live vLLM inference pod via an `ncu` debug sidecar (no production image rebuild). Outputs `.ncu-rep` + per-kernel CSV. Scoped via `--kernel-name` to avoid the 10-100x kernel-replay slowdown that an unscoped ncu attach would impose on serving. Pairs with `inference-kernel-profile` (nsys: absolute kernel duration + cuda-graph timeline) and `zymtrace-anchored-query` (sample- share + kernel-name resolution). Triggers on "ncu", "nsight compute", "kernel internals", "occupancy", "register pressure", "shared memory", "DRAM bandwidth", "roofline", "arithmetic intensity", "warp stalls", "branch divergence", or any combination of "ncu / nsight-compute / occupancy / regs / smem / dram / roofline" with "vllm / kimi / glm / deepseek / inference".

0
installs
▌
Inference Quantize Calibrate · cfregly
Produce quantized inference weights from a BF16/FP8 base checkpoint via a post-training-quantization (PTQ) pipeline -- instead of only ever pulling NVFP4 weights pre-quantized. A calibration prep Job then a quantize Job: NVIDIA ModelOpt -> NVFP4 (default), or llm-compressor -> FP8 (alt). Writes weights + per-layer scales to an experiment-prefixed PVC, gated on inference-model-eval accuracy + a perf A/B. When a vendor nvidia/*-NVFP4 exists, runs a 3-way comparison (base vs ours vs NVIDIA) on accuracy + perf + ncu. Ack-gated. The calibrate+quantize phase of inference-model-optimize. Triggers on "quantize a model", "calibrate NVFP4", "run modelopt PTQ", "produce NVFP4 weights", "llm-compressor FP8", "PTQ pipeline", "compare our PTQ to NVIDIA NVFP4", "our nvfp4 vs nvidia nvfp4", "calibration dataset for quantization", "quantize <model> to NVFP4/FP8", or any combination of "quantize / calibrate / PTQ / modelopt / llm-compressor" with "nvfp4 / fp8 / weights / model / inference".

0
installs
▌
Inference Spec Decode Service · cfregly
Coordinate a measured speculative decoding workflow for a served model. Profile representative traffic, build a matched corpus, run an operator-provided draft trainer, compare the draft with the standing config, and promote only a measured win. Use for profile-matched speculative decoding work that needs one evidence trail and fail-closed gates.

0
installs
▌
Inference Perf Baseline Bridge · cfregly
Bridge between the inference-perf-bench / ai-bench output bundle and the profile-and-optimize perf-baseline registry. Knows the canonical inference perf metric shape (TTFT p50/p95/p99, ITL p50, throughput, tok/s/user, request latency p50, prefix cache hit rate delta, GPU cache utilization peak), records it under inference_perfbench_v1 schema, and on diff applies per-metric tolerances tuned for inference workloads (TTFT regressions <5 percent, throughput <3 percent, cache hit rate <2 absolute points). Wraps the existing perf-baseline-record and perf-baseline-diff MCP verbs, no new MCP verbs introduced. Triggers on "register inference baseline", "inference perf-baseline", "diff inference perf", "is this inference regression real", "perf-bench baseline", "ai-bench baseline", or any combination of "register / record / diff / compare" with "inference / ai-bench / perf-bench / vllm / kimi / glm / deepseek / minimax".

0
installs
▌
Inference Kernel Whitebox Debug · cfregly
White-box debug a custom CUDA/CUTLASS kernel producing a WRONG numeric result (over-amplification, NaN, coherence) after black-box bisection is EXHAUSTED: every external lever varied (grid, pipeline depth, layout, dtype), the result deterministic + lever-independent, yet the structure reads correct. Two tracks: (A) an in-kernel operand + accumulator trace (`if constexpr`-gated to survive megakernel pipelining), (B) a standalone reproducer .cu instantiating the EXACT kernel template + TMA descriptors with controlled inputs (all-ones then real-dump) vs a host GEMM. Localizes the defect to operand-load vs MMA vs epilogue vs descriptor. Escalation tier above `inference-kernel-ncu-profile` when H/P are fine but the output is numerically wrong. Triggers on "white-box kernel", "kernel over-amplifies / NaN", "standalone reproducer", "in-kernel trace", "operand dump", "cutlass UMMA bug", or any combination of "white-box / reproducer / in-kernel / operand-trace" with "kernel / cutlass / UMMA / megakernel / vllm".

0
installs

35 published skills