Results for “nlp-metrics”

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Build the funnel metrics that actually get trusted. Stage-by-stage conversion, velocity, win rate, and the single biggest leak, with every definition pinned so nobody relitigates the numbers in the meeting. Built for B2B RevOps teams, customizable to your CRM and your stage model. Trigger on "build my funnel metrics", "what's my conversion by stage", "where's the leak", "what's our win rate", "how fast do deals move", or any funnel diagnostic.

Fast tokenization for NLP using Rust-based tokenizers supporting BPE, WordPiece, and Unigram algorithms, with training, alignment tracking, and padding/truncation.

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

Solve LP, MILP, and QP problems using the cuOpt C API with a consistent build pattern and core calls.

Evaluates LLM performance using BLEU, ROUGE metrics and LLM-as-judge. Use for model testing.

Computes the Recall metric using torchmetrics, including configuration for binary, multiclass, and multilabel tasks.

Audits medical LLM benchmarks across five lifecycle phases using 46 medically tailored criteria to assess clinical relevance, data integrity, safety-critical capabilities, validity, and governance.

Track ML experiments with automatic logging, visualize training in real-time, optimize hyperparameters with sweeps, and manage model registry with W&B.

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution on local Docker, Slurm HPC, or cloud platforms.

Evaluates text-to-speech systems by measuring distributional distance between synthetic and real speech across five factors, producing a scalar score without subjective MOS ratings.

Track ML experiments with automatic logging, visualize training in real-time, optimize hyperparameters with sweeps, and manage model registry with W&B - collaborative MLOps platform

Designing and evaluating AI/ML systems: prompt engineering, RAG design, LLM application patterns, AI safety, evaluation frameworks, MLOps, cost optimization. Use for AI pipelines or eval harnesses.

Every metric encodes an opinion about which mistake hurts.

Explain and apply core ML/AI concepts — model types, training pipelines, evaluation metrics, and neural architectures.

Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking

This skill should be used when the user asks about "key startup metrics", "SaaS metrics", "CAC and LTV", "unit economics", "burn multiple", "rule of 40", "marketplace metrics", or requests guidance on tracking and optimizing business performance metrics.

---

Guides analysis of Neuropixels recordings from raw data to curated units, covering preprocessing, motion correction, spike sorting, quality metrics, and export.

Track ML experiments with automatic logging, visualize training in real-time, optimize hyperparameters with sweeps, and manage model registry with W&B - collaborative MLOps platform

Computes the InfoLM metric from torchmetrics for evaluating text generation against ground truth, with configurable information measures and sentence-level scoring.