Results for “l-eval”

Guides production LLM operations: RAG pipelines, embeddings, vector databases, fine-tuning, prompt engineering, cost estimation, quality evals, and AI architectures.

Use this skill when creating a new Claude skill from scratch, editing or improving an existing skill, or measuring skill performance with evals and benchmarks. Invoke whenever the user says things like 'make a skill for X', 'turn this workflow into a skill', 'test my skill', 'improve my skill', 'run evals', 'benchmark this', or 'optimize my skill description'. Also use proactively when the conversation has produced a repeatable workflow that would benefit from being captured as a skill. Covers the full lifecycle: capture intent, draft SKILL.md, run evals, review with user, iterate, optimize description, package. NOT for general coding help, debugging runtime errors, building MCP servers, writing Claude hooks, or creating plugins - use domain-specific skills for those.

Train, evaluate, export, prune, and run inference for PointPillars 3D object detection models from LiDAR point clouds using NVIDIA TAO.

Provides guidance on production AI operations including RAG pipelines, vector databases, embeddings, fine-tuning, prompt engineering, cost estimation, and quality evaluation.

Fine-tune a TAO model with standard supervised training, evaluation, and export, with AutoML bypass and platform-specific credential intake.

Guides production AI systems: RAG pipelines, embeddings, vector databases, fine-tuning, prompt engineering, cost estimation, quality evals, and caching.

Turns model work into a production ML system with data contracts, reproducible training, quality gates, deployable artifacts, and monitoring.

Evaluate RAG pipelines with Ragas — measure faithfulness, answer relevancy, context precision/recall, and noise sensitivity using LLM-as-judge metrics; run automated test suite generation with TestsetGenerator; integrate with LangChain, LlamaIndex, and CI pipelines.

Analiza series temporales: tendencia, estacionalidad y pronóstico con Prophet, statsmodels y ML, incluyendo descomposición, tests de estacionariedad y evaluación contra baselines.

Evaluates the agent's own work, catches mistakes, and improves permanently through self-reflection, self-criticism, and learning from corrections.

Optimizes long-running agent sessions with structured context compression, summarization, and durable handoff summaries that preserve decisions, files, risks, and next actions.

Designs and implements RAG pipelines, covering document chunking, embedding strategies, hybrid search, answer synthesis with source attribution, and evaluation using RAGAS metrics.

Expert in designing effective prompts for LLM-powered applications. Masters prompt structure, context management, output formatting, and prompt evaluation. Use when: prompt engineering, system prompt, few-shot, chain of thought, prompt design.

Route Firebase AI feature work into either direct app/client Firebase AI Logic SDK integration or a server-owned Genkit workflow. Use when a web, mobile, backend, or full-stack feature needs model calls, typed outputs, reusable flows, tools, retrieval, prompt files, evals, observability, or deployment. Choose client-ai-logic, flow-foundation, tool-and-agent, retrieval-and-prompt, evaluation-and-observability, deployment-runtime, or comparison-or-fallback; route Firebase platform/operator work to `firebase-cli` and broad framework comparisons to `survey`.

Entrena modelos de machine learning con Scikit-learn, LightGBM y XGBoost, desde un baseline hasta un modelo productivo con validación robusta y explicabilidad.

You are an expert in LLM observability and evaluation. You think in terms of traces, spans, and metrics. You know that LLM applications need monitoring just like traditional software - but with different dimensions (cost, quality, latency).

Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo.

Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo.

Implements production LLM operations: RAG pipelines, embeddings, vector databases, fine-tuning, advanced prompt engineering, cost estimation, quality evals, semantic caching, streaming, and agents.

Turn model work into a production ML system with data contracts, repeatable training, measurable quality gates, deployable artifacts, and operational monitoring.

Turn model work into a production ML system with data contracts, reproducible training, quality gates, deployable artifacts, and monitoring.

Guides the creation of high-quality MCP servers, covering design, implementation, testing, and evaluation for integrating external services with LLMs.

Audits medical LLM benchmarks across five lifecycle phases using 46 medically tailored criteria to assess clinical relevance, data integrity, safety-critical capabilities, validity, and governance.

You are an expert in LLM observability and evaluation. You think in terms of traces, spans, and metrics. You know that LLM applications need monitoring just like traditional software - but with different dimensions (cost, quality, latency).

Creates and manages annotation configs and annotation queues on Arize, and applies human annotations to project spans via the Python SDK.

Reference guide for evaluating LLM-generated translations between CUDA and OpenMP, covering memory model mapping, kernel launch patterns, shared memory, atomics, and common failure modes.

Expert skill for building LLM applications with the LlamaIndex framework — RAG pipelines, multi-agent orchestration, event-driven workflows, knowledge graph construction, production deployment, and evaluation. Use when working with LlamaIndex or comparing RAG and agent orchestration frameworks.

Curates clinical-specialty term lists, generates IPA-tagged synthetic audio via TTS, and produces NeMo-format manifests for ASR benchmark evaluation.

Benchmarks medical AI agents on synthetic EHR tasks, measuring success rates for data retrieval, temporal aggregation, and threshold-based conditional logic with exact-match scoring.

Submit and manage TAO training, evaluation, and inference jobs on SLURM GPU clusters over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed storage.

Collects launch inputs and runs preflight checks before executing TAO workflows such as AutoML, training, evaluation, inference, export, TensorRT engine generation, or DEFT jobs on supported platforms.

Trains, evaluates, exports, and runs inference for FastFoundationStereo (FFS) stereo depth estimation models, a distilled variant of FoundationStereo with lower latency.

Require the learner to explain a concept in their own words before the AI evaluates or extends it. Ensures the AI works from the learner's understanding rather than providing an explanation from scratch.

BOB (Build on Bitcoin): hybrid Bitcoin/Ethereum L2 using OP Stack + Bitcoin security. Bridges to Bitcoin via threshold signatures + zk-proofs. USE WHEN: building cross-chain Bitcoin/Ethereum apps, evaluating BOB hybrid model.

Provides guidance and code for production AI workflows including RAG pipelines, vector databases, embedding indexing, prompt engineering, cost estimation, semantic caching, and quality evaluation.

Quantifies audio source separation quality by computing the signal-to-distortion ratio (SDR) between ground-truth and estimated stems, with per-stem and record-level averaging.