Traigent
- 18 skills
- 0 followers
- 6 hours ago last updated
- ▌ Traigent JS · traigent bundleSet up and run native JavaScript/TypeScript optimization with @traigent/sdk. Use when a user asks to optimize a JS/TS agent function, use optimize(spec)(agentFn), configure param.* search spaces, define evaluation.data/loadData metrics, use getTrialParam/getTrialConfig, or author backend-routed config-space specs for Traigent-compatible services.
- ▌ Traigent Debugging · traigent bundleDebug and troubleshoot Traigent optimization issues. Use when encountering CostLimitExceeded, ConfigurationError, OptimizationStateError, ModuleNotFoundError, or when optimization produces unexpected results. Covers mock mode, logging configuration, and common error resolution.
- ▌ Traigent Eval Audit · traigent bundleAudit evaluator reliability before trusting Traigent optimization decisions. Use when users ask: is my LLM judge reliable, audit my evaluator, judge agreement, evaluator calibration, calibrate thresholds, parse-failure policy, repeated-judge stability, bias probes, or when optimization results depend on an LLM-as-judge metric.
- ▌ Traigent Eval Build · traigent bundleBuild Traigent evaluators and scoring code. Use when wiring eval_dataset, scoring_function, metric_functions, custom_evaluator, ExampleResult, BaseEvaluator subclasses, deterministic checks, LLM judges, statistical repeated evaluations, hybrid evaluators, or fixing tuple-return/custom-scorer pitfalls.
- ▌ Traigent Boost Agent · traigent bundleEnd-to-end lifecycle playbook — from a single decorated function to a full 12-step codebase onboarding — for adding Traigent to an existing client agent codebase and measurably boosting accuracy, cost, latency, or reliability. Use when asked to add Traigent to an existing agent codebase, onboard this agent to Traigent end-to-end, run a full agent-build lifecycle, wire an evaluator and optimize for an agent with code to instrument, boost accuracy/cost of an existing agent, select TVARs with generate_config() on agent code, choose composite knobs by agent shape, instrument @traigent.optimize into agent code, validate in mock mode, run real optimization with budgets, inspect results, iterate to refine an existing agent, or gate a promoted config. For brand-new users without an existing agent codebase, see traigent-setup-quickstart. ALWAYS start with dry-run (mock mode) to validate the full pipeline, then switch to real execution only when the user explicitly requests it.
- ▌ Traigent Optimize Run · traigent bundleRun Traigent optimization: async/sync execution, algorithm selection, cost limits, stop conditions, and parallel trials. Use when calling func.optimize() or optimize_sync(), choosing algorithms (auto/grid/random anywhere; named smart selectors like bayesian/optuna on authenticated connected runs, SDK 0.20.1+), setting max_trials or cost_limit, configuring parallel execution, or handling CostLimitExceeded.
- ▌ Traigent CI Safety Gate · traigent bundleAdd Traigent safety and promotion gates to CI. Use when users ask to add safety constraints, gate the optimized config, prevent regressions in CI, enforce cost or latency budgets, compare candidate versus incumbent, validate TVL specs, or write GitHub Actions for agent optimization safety.
- ▌ Traigent Dataset Curate · traigent bundleCreate and improve a Traigent evaluation dataset / JSONL eval set. Use when asked to create an evaluation dataset, check whether examples are good enough, synthesize more examples, grow a dataset, score examples after a run, inspect dataset quality, design a holdout split, avoid leakage in eval data, reflect on hard examples, work through server-flagged hard or broken example IDs, map flagged example IDs to local content, or run the content-reflection loop after a run.
- ▌ Traigent Analyze Results · traigent bundleAnalyze and report Traigent optimization results from the terminal — without opening the portal's tabs. Use when a user asks to analyze a run, 'how did my run do?', 'analyze my latest run in project X', what the winner is, or to read result fields, reports, leaderboards, Pareto trade-offs, correlations, or parameter/example insights. Decision questions route to `traigent-analyze-guidance` for portal-tracked runs and `traigent-analyze-guidance` for offline/local runs. Also covers the local OptimizationResult object: reading results.best_config, comparing trials, checking stop_reason, calling apply_best_config(), accessing total_cost or total_tokens, or understanding why optimization stopped.
- ▌ Traigent Recipe Text2sql · traigent bundleEnd-to-end recipe to optimize a text2SQL agent with Traigent and reach high accuracy at low cost. Use when wiring a SPIDER-style NL->SQL agent with @traigent.optimize: execution-match scoring, model + structural knobs, weighted ACL objectives, mock dry-run, then a real portal-tracked run. Captures the working configuration that took a plain agent from 66.7% -> 90% on the cheap model.
- ▌ Traigent Setup Decorator · traigent bundleConfigure the @traigent.optimize() decorator with evaluation, injection, and execution options. Use when setting up eval_dataset, choosing injection_mode, choosing the optimization algorithm or offline execution, defining objectives, naming an optimization with experiment_name (an agent identity key, not a per-run label; there is no tags/metadata argument), using EvaluationOptions/InjectionOptions/ExecutionOptions, or integrating custom evaluators. Provide the agent function + its path, an eval dataset, and the objective(s).
- ▌ Traigent Analyze Guidance · traigent bundleWhat should this Traigent optimization run be, and what next? Three modes: (A) pre-run — fetch the service run-plan, present objectives/models/knobs/search/budget/offline options, apply preflight; (B) post-run, portal-tracked — fetch the backend decision brief via the `analytics_get_run_decision_brief` MCP tool, present its headline, confidence, recommended action, evidence, and warnings verbatim, then hand off to the matching skill; (C) offline/local fallback — diagnose flat/noisy/negative local results, which knob mattered, example evidence, form the next iteration hypothesis when offline=True or no service payload. Portal-tracked decisions come from Traigent, never local markdown.
- ▌ Traigent Setup Quickstart · traigent bundleInstall, set up, and get first value from the Traigent SDK for LLM optimization. The cold-start path: use when the user is new to traigent, wants their first run, has no dataset yet, or wants to install traigent, set up their first optimization, create an evaluation dataset, or get started with @traigent.optimize. Covers pip install, API-key setup, mock mode, a linear first-value walkthrough, and running a first optimization.
- ▌ Traigent Eval Choose Metric · traigent bundleChoose Traigent objectives and metric functions before optimizing. Use when asked which metric to use, how to measure quality, whether to optimize accuracy/cost/latency/safety, how to name objectives, how to combine multiple objectives, when to use custom metric_functions, or how to turn product goals into Traigent objectives.
- ▌ Traigent Setup Integrations · traigent bundleIntegrate Traigent with LangChain, LiteLLM, DSPy, and other AI frameworks. Use when importing langchain/litellm/dspy alongside traigent, setting up multi-provider model testing, using auto_override_frameworks, or asking about framework-specific adapter patterns.
- ▌ Traigent Optimize Config Space · traigent bundleDefine tuned variables, structural knobs, and configuration spaces for Traigent optimization. Use when setting up parameter search spaces, choosing models/temperatures/prompts, designing task-level text2SQL/RAG/multi-hop knobs, using Range/IntRange/Choices/LogRange types, adding constraints, or using factory presets like Range.temperature().
- ▌ Traigent Optimize Composite Knobs · traigent bundleDeclare and run Traigent composite knobs: cascades, routers, ensembles, self-consistency, best-of-n, self-refine, self-debug, ReAct tool loops, verification gates, mixture-of-experts, and fallback patterns. Use when choosing a catalog pattern, assessing whether cheap-to-strong routing or escalation is fit for a task, defining a comparator or gate signal, calibrating thresholds, wiring StageRunner/LoopBodyRunner execution, merging composite telemetry into metrics, or explaining calibration-backed claim scope.
- ▌ Traigent Analyze Variable Importance · traigent bundleShow significant tuned variables and rank which variables mattered in a Traigent optimization. Use for: show significant tuned variables, which variables mattered, tuned variable importance, feature importance for optimization, optimization gains attribution, parameter importance with honest confidence labels, or one-glance video card summaries of what drove optimization gains.