Results for “ab-testing”

Audits AGENTS.md and CLAUDE.md files for bloat, redundancy, and ineffective rules, then runs A/B evals to prove which instructions are dead weight.

Structured guide for setting up A/B tests with mandatory gates for hypothesis, metrics, and execution readiness.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Guides statistical hypothesis testing with assumption checks, effect sizes, power analysis, Bayesian alternatives, and APA-formatted reporting for research data.

Test vector stores for embedding inversion, cross-tenant leakage, and poisoning.

Benchmarks LLM bias by scoring responses to automatically generated open-ended questions across sensitive attributes, producing a composite fitness score from 0 to 5.

Benchmarks large language models on symbolic music understanding and instruction following using text-based ABC notation, covering syntax parsing, error detection, segment-level reasoning, and sequence-level musical analysis.

Prevents AI-introduced regressions with sandbox-mode API testing, automated bug-check workflows, and patterns that catch blind spots where the same model writes and reviews code.

After scaffolded practice, run an unassisted check — a problem with no AI help. Separates what the learner can do with support from what they can do independently. Critical for preventing phantom attainment.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality.

Create and troubleshoot Ocana agents. Use when creating a new Ocana agent, refreshing/repairing management token auth, diagnosing `ocana_agent_create` failures, checking `gatewayReachable`, handling agents stuck in `provisioning`, debugging `226/NAMESPACE` or missing `/usr/bin/openclaw`, or testing lifecycle/reprovision flows for a newly created agent.

Evaluates the correlation between a zero-cost NAS metric (epsilon) and actual training accuracy across different neural architecture search spaces, testing the metric's ability to rank architectures without training. It probes whether output dispersion from constant weight initializations can serve as a reliable.

Manual for the marginaleffects R and Python package, and guide to the book "Model to Meaning". Use when users ask about predictions, comparisons, slopes, marginal effects, average treatment effects (ATE/ATT/CATE), hypothesis testing, contrasts, counterfactuals, risk ratios, odds ratios, causal inference with G-computation, or need help with marginaleffects functions like predictions(), comparisons(), slopes(), hypotheses(), datagrid(), avg_predictions(), avg_comparisons(), avg_slopes(), or plot functions.

Use ONLY for code review and test write/review guidance in `packages/agent-core-v2` (the DI × Scope agent engine). Does NOT apply to the legacy `packages/agent-core` or to any other package — for those, do not load this skill. Groups the review and testing lenses used for agent-core-v2 — `slop` (single-level-of-abstraction / layered error-handling review, invoked only on explicit request) and `test` (contract-driven per-test rules for both authoring and reviewing tests). Apply the sub-skill that matches the task; do not apply `slop` unprompted.