Results for “l-eval”

Perform survival analysis and time-to-event modeling in Python using scikit-survival, including Cox models, random survival forests, gradient boosting, survival SVMs, and evaluation metrics like concordance index and Brier score.

Build multi-agent AI systems with LangGraph — the low-level orchestration framework for stateful, graph-based agent workflows. Covers supervisor, swarm, and hierarchical multi-agent patterns; subgraph composition; state management (checkpointers/stores); persistence; evals; and production debugging. Reach for this when designing agent architectures that need cycles, conditional branching, parallel execution, or human-in-the-loop patterns.

Use when the user wants to train, compare, or export machine learning models using Classification Learner or Regression Learner — including opening the app, loading data, training models, evaluating metrics, comparing results, visualizing plots, testing on held-out data, exploring model interpretability, and exporting trained models. Programmatic access to Classification Learner and Regression Learner apps via AppController.

Remove refusal behaviors from open-weight LLMs using OBLITERATUS — mechanistic interpretability techniques (diff-in-means, SVD, whitened SVD, LEACE, SAE decomposition, etc.) to excise guardrails while preserving reasoning. 9 CLI methods, 28 analysis modules, 116 model presets across 5 compute tiers, tournament evaluation, and telemetry-driven recommendations. Use when a user wants to uncensor, abliterate, or remove refusal from an LLM.

Fine-tunes and evaluates OpenVLA-OFT and OpenVLA-OFT+ policies for robot action generation with continuous action heads, LoRA adaptation, and FiLM conditioning on LIBERO simulation and ALOHA real-world setups. Use when reproducing OpenVLA-OFT paper results, training custom VLA action heads (L1 or diffusion), deploying server-client inference for ALOHA, or debugging normalization, LoRA merge, and cross-GPU issues.

Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation. Agents work independently, results are evaluated by metric or LLM judge, and the best branch is merged. Use when: user wants multiple approaches tried in parallel — code optimization, content variation, research exploration, or any task that benefits from parallel competition. Requires: a git repo.

Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation. Agents work independently, results are evaluated by metric or LLM judge, and the best branch is merged. Use when: user wants multiple approaches tried in parallel — code optimization, content variation, research exploration, or any task that benefits from parallel competition. Requires: a git repo.

Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation. Agents work independently, results are evaluated by metric or LLM judge, and the best branch is merged. Use when: user wants multiple approaches tried in parallel — code optimization, content variation, research exploration, or any task that benefits from parallel competition. Requires: a git repo.

Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation. Agents work independently, results are evaluated by metric or LLM judge, and the best branch is merged. Use when: user wants multiple approaches tried in parallel — code optimization, content variation, research exploration, or any task that benefits from parallel competition. Requires: a git repo.

Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation. Agents work independently, results are evaluated by metric or LLM judge, and the best branch is merged. Use when: user wants multiple approaches tried in parallel — code optimization, content variation, research exploration, or any task that benefits from parallel competition. Requires: a git repo.

Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation. Agents work independently, results are evaluated by metric or LLM judge, and the best branch is merged. Use when: user wants multiple approaches tried in parallel — code optimization, content variation, research exploration, or any task that benefits from parallel competition. Requires: a git repo.

Give your agent a budget, a target, and a deadline — it does the rest. Orchestrates DSL + Opportunity Scanner + Emerging Movers into a full autonomous trading loop on Hyperliquid. Race condition prevention, conviction collapse cuts, cross-margin buffer math, speed filter. 3 risk profiles: conservative, moderate, aggressive. Use when setting up autonomous trading, creating a trading strategy, or running a scan-evaluate-trade-protect loop.

4-stage funnel that screens all 500+ Hyperliquid perps down to the top trading opportunities. Scores setups 0-400 across smart money, market structure, technicals, and funding. BTC macro filter, hourly trend gate (counter-trend = hard skip), cross-scan momentum tracking. Near-zero LLM tokens — all computation in Python. Use when scanning for new trading opportunities on Hyperliquid, evaluating setups, or checking market conditions.

Debug LLM applications using the Phoenix CLI. Fetch traces, analyze errors, structure trace review with open coding and axial coding, inspect datasets, review experiments, query annotation configs, and use the GraphQL API. Use whenever the user is analyzing traces or spans, investigating LLM/agent failures, deciding what to do after instrumenting an app, building failure taxonomies, choosing what evals to write, or asking "what's going wrong", "what kinds of mistakes", or "where do I focus" — even without naming a technique.

Design habit-forming product loops using the Hook Model (Trigger, Action, Variable Reward, Investment). Use when the user mentions "users arent coming back", "engagement loops", "habit formation", "push notifications", "variable rewards", "daily active users", "habit zone", or "user retention loops". Also trigger when designing notification strategies, building streaks or progress systems, or analyzing why users stop using a product after initial signup. Covers ethics evaluation and onboarding for habits. For friction reduction and B=MAP, see improve-retention. For viral sharing, see contagious.

Read-only diagnostic that scans Claude Code session transcripts to surface which MCP servers and tools you actually use, ranked by call frequency, with zero-use servers flagged for removal. Use when planning an MCP cleanup, evaluating whether to keep a newly-added server, deciding which servers warrant token cost in the catalog, or before authoring an MCP-removal PR. Outputs a markdown report (last N days) — does not modify any settings. Pair with the manual `claude mcp remove <name>` step once findings are reviewed.

Guardian Angel gives AI agents a moral conscience rooted in Thomistic virtue ethics. Rather than relying solely on rule lists, it cultivates stable virtuous dispositions— prudence, justice, fortitude, temperance—that guide every interaction. The foundation is caritas: willing the good of the person you serve. From this flow the cardinal virtues as practical habits of right action and sound judgment. v3.0 introduced virtue-based disposition as the primary evaluation layer, providing deeper coherence than checklists alone. The agent's character becomes the safeguard. v3.1 adds: Plugin enforcement layer with before_tool_call hooks, approval workflows for ambiguous cases, and protections for sensitive infrastructure actions.

Drive Soup (`soup-cli`), a CLI-first tool for fine-tuning and post-training LLMs with one YAML config and one command — SFT, DPO/GRPO/ORPO/SimPO/KTO, QLoRA/DoRA/LoRA+, layer streaming for 4-8 GB GPUs, eval-gated training, and serving. Use when the user wants to `soup init`/`soup train` a model, pick a training method or quantization scheme, estimate cost/memory before training, fine-tune on a small local GPU, migrate a config from Axolotl/LLaMA-Factory/Unsloth, or serve/merge/push a trained adapter. Triggers on: "soup-cli", "soup train", "soup init", "fine-tune an LLM locally", "QLoRA on a laptop GPU", "layer streaming", "soup advise", "soup autopilot", "DPO/GRPO/ORPO training", "merge LoRA adapter".

Instrument a shipped product's AI agents with tracing and observability so you can see what they did, why outputs happened, and what each run cost. Plain-language primer plus free-tier-first backend selection (Langfuse, Phoenix, LangSmith, Braintrust) and OpenTelemetry/OpenInference instrumentation. Load when the user asks to add observability, add tracing, instrument my agents, see what my agent is doing in production, set up Langfuse or Phoenix or LangSmith, debug why my agent gave a bad answer, or track LLM cost per request. Also fires when agent-system-architecture or setup-evaluation requires an observability plan for an agent-chain product. NOT for tracing the coding agent itself — that is run-trace. Precondition for runtime-learning-loop.

Run the Ouroboros specification-first development loop: reduce ambiguity with a Socratic interview grounded in live git data (commits, churn, contributors), freeze an immutable seed/spec, render the execution plan through spec-kit (/speckit.plan → /speckit.tasks), execute against that contract through cli-anything agent-native CLI harnesses (cli-hub, --json evidence), verify before claiming success, and keep looping until completion is actually verified. Use when the user wants spec-first clarification, git-aware interviews, immutable requirements, drift-aware implementation, harness-driven execution, or a persistent completion loop that should keep going until tests / checks / acceptance criteria pass. Triggers on: ooo, ouroboros, interview, seed, run workflow, evaluate, evolve, ooo ralph, specification first, socratic interview, git-aware interview, ambiguity reduction, execution plan, cli harness execute, persistent completion.

Use this skill to classify tabular data end-to-end in MATLAB — load a dataset, prepare and clean it, select promising classifiers, train them, and compare accuracies with cross-validation, holdout, or hyperparameter optimization plus statistical tests. TRIGGER when: user asks to classify tabular data, pick classifiers for a dataset, compare classifier accuracy, run cross-validation or a holdout evaluation, or find the best model with statistical uncertainty. DO NOT TRIGGER when: user has non-tabular inputs (images, sequences, time series), wants a regression model, is training a specific neural network architecture (use matlab-train-network), or wants cost-sensitive learning or an arbitrary class-prior vector (this skill only supports the built-in uniform-prior toggle for imbalanced data).

Use BEFORE generating, refactoring, reviewing, or debugging code. Trigger phrases include "write a function/script/class for X", "review this code/diff/PR", "refactor this", "debug this error", "is this implementation correct", "what's wrong with this code", "improve this code", "translate from X to Y", or any prompt with a code block the user wants you to act on. Also fires when planning architectural changes, picking algorithms or data structures, or evaluating dependency upgrades. Calls the code MCP tool to retrieve an engineering scaffold (failure pattern, procedure, correct-pattern example, verification step) before generating. Catches hallucinated APIs, lost edge cases, premature algorithm commitment, silent contract violations, refactors that change behavior masked by passing tests. Do NOT trigger for pure code reading with no action requested, simple syntax questions, file...

AI/LLM red-team assessment using the OWASP LLM Top 10 (2025) + OWASP AI Testing Guide (AITG v1, Nov 2025) frameworks, plus OWASP MCP Top 10 runtime testing for agentic/MCP targets. Tests prompt injection, jailbreaks, system prompt leakage, sensitive data extraction, excessive agency, improper output handling, model extraction, content bias, evasion, membership inference, MCP token exposure, MCP command injection, and more. Uses four tools in combination: FuzzyAI (single-turn jailbreak fuzzing), PyRIT (multi-turn orchestrated attacks), Garak (probe-based vulnerability scanning), and promptfoo (plugin-based red-team evaluation). Each tool covers different OWASP categories; running them together gives systematic coverage. Includes a conditional MCP reconnaissance phase and a post-access AI infrastructure phase (chained from /post-exploit). Produces: OWASP LLM Top 10 + AITG + MCP coverage matrix, findings per category, architecture diagram of the AI system, PoCs for confirmed exploits. Chains into /gh-export for

Expert strategy advisor for products built on LLMs or agents — not general product strategy (see `product-strategy` for that). Use this — proactively and without waiting to be asked — whenever choosing where to apply AI in a product, deciding between RAG and fine-tuning, designing how much autonomy an AI feature should have, evaluating whether an AI feature is actually defensible, or deciding whether to add AI to a feature at all. Also triggers for: "should this be an agent or a simple LLM call", "how much autonomy should this feature have", "RAG vs fine-tuning", "is this AI feature defensible", "our AI feature keeps hallucinating and users don't trust it", "should we build this with AI or just ship it deterministic", "AI product wedge", "what happens to this feature when the models get better", "human-in-the-loop design for AI features". Produces a decision-focused brief: the wedge, the architecture choice, the autonomy level, and the defensibility bet — each with an explicit trade-off.