Results for “evaluation”

Benchmarks zero-shot in-context learning on BIG-Bench Hard multiple-choice tasks, comparing self-generated demonstrations against direct prompting and chain-of-thought baselines, and reports accuracy.

High-performance DataFrame library for Python ETL, analytics, and pandas migration. Use for expression-based data manipulation with lazy query optimization, parallel execution, streaming out-of-core processing, Arrow interoperability, and optional GPU execution.

Evaluates image captions by converting them into scene graphs and computing an F-score over semantic propositions, measuring how well a generated caption captures the meaning of an image compared to human references.

Benchmarks energy-function-based safe control algorithms on the BIS (Benchmark of Interactive Safety) dataset, scoring safety, efficiency, and hybrid performance in human-robot and robot co-working scenarios.

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

Documents MariaDB-specific UPDATE syntax and behavior, including single-table vs multi-table forms, assignment evaluation order, RETURNING with OLD_VALUE(), temporal tables, and common pitfalls. Use when writing, generating, or reviewing UPDATE statements targeting MariaDB.

Use when planning an ICSE research-track campaign end to end, covering the single-cycle calendar from the June abstract deadline through September response, October decisions, the November revision sprint, artifact evaluation, camera-ready, and the April conference, plus rerouting timelines after rejection.

Use when planning a POPL campaign calendar — backward-scheduling theory and mechanization from the July deadline, riding the October notification into the conditional-acceptance revision and artifact evaluation, landing the January conference, and retargeting across the SIGPLAN/PACMPL deadline wheel if July slips.

Use when planning a SOSP campaign across the annual cycle — back-scheduling from the spring abstract and paper deadlines, running the summer response and July notification, stacking the August camera-ready with artifact evaluation, and retargeting rejections across the SOSP/OSDI/EuroSys circuit.

Design, implement, and validate reproducible machine-learning pipelines spanning data preparation, training, evaluation, registry, and deployment gates. Use when the user requests an ML pipeline, needs to turn model scripts into an orchestrated workflow, or provides pipeline components that must be connected safely.

Foundational Clojure patterns covering functional programming, REPL-driven development, immutable data structures, and idiomatic code. Use when writing Clojure code, working with sequences and lazy evaluation, understanding macros, or needing guidance on functional programming patterns. This is the entry point for Clojure development.

Provides domain knowledge of Korean government and private funding program announcement systems, evaluation criteria, and key program characteristics, including TIPS, Early-Stage Startup Package, Startup Growth Technology Development, and AI Voucher.

Evaluates text-to-image generation models by decomposing assessment into five specialized skills (object presence, count, spatial relations, scale, and text rendering) and open-ended prompts, producing interpretable binary scores with visual and textual explanations.

Evaluates speech language models on beyond-semantic speech attributes such as dialect comprehension, multi-turn context memory, emotion perception, age-aware response generation, and non-verbal cue handling, reporting accuracy and judge-based scores.

Evaluate everything the PA agent manages — tasks, skills, PA network health, billing, calendar connections, and memory quality. Use when: owner asks for an evaluation, wants to know what's working and what isn't, or requests a performance report. Combines supervisor status with quality scoring.

Deterministically score, route, rank, and select at most one fully vetted Pleasur.ai Stage 01 blog-keyword candidate using product-fit-dominant business value, traffic opportunity, brand fit, DR-relative winnability, and a free-seeker penalty. Use only after BID and AIO evaluation are complete.

Drafts evaluation-ready U.S. federal RFP responses across all standard proposal volumes (cover letter, technical, cost/price, reps and certs). Enforces FAR compliance, CPARS references, and Section L/M alignment. Use when preparing federal solicitation submissions, responding to government RFPs, or drafting procurement bids.

Use when the request is broad and spans idea evaluation, MVP planning, UX, frontend, backend, database, auth, integrations, and launch strategy. Trigger on asks like "build this app", "turn this idea into a product", "design the whole system", or "plan everything end to end".

Provides a formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles to define pass/fail criteria, measure reliability with pass@k metrics, and create regression test suites.

Evaluate scientific claims and evidence quality by assessing experimental design, identifying biases and confounders, and applying evidence grading frameworks like GRADE and Cochrane Risk of Bias.

Automates detection engineering workflows in Google SecOps by extracting threat intelligence, generating detection opportunities, simulating attacker behavior with synthetic events, evaluating rule coverage, and creating new YARA-L 2.0 rules to close gaps.

Automates the full DEFT AOI improvement loop for NVIDIA TAO VisualChangeNet / ChangeNet PCB inspection models, including baseline evaluation, RCA, synthetic defect generation, data mining, retraining, and deployment gating until KPI targets are met.

Use when designing or auditing the evaluation of an ASE (IEEE/ACM Automated Software Engineering) paper, covering real subject systems, fair runnable tool baselines, task-matched effectiveness metrics, ablations that isolate a learned component, oracle and correctness validation, contamination-aware LLM handling, and provenance for mining.

Conversion Rate Optimization auditing for landing pages, signup flows, checkout funnels, forms, and CTAs. Identifies friction points, runs heuristic evaluations, produces prioritized recommendations using ICE/PIE frameworks. Use when the user asks about conversion optimization, funnel analysis, landing page audits, form optimization, CTA testing, or checkout improvement.

ML / research person pack for Content Factory. Use when the user says person ml, @person-ml, ML person, research scientist person, or LLM researcher person. Auto-loads ml-research-engineer, llm-researcher, ai-research-scientist plus experiment-tracking, evaluation, cf-llm-model-usage.

Evaluates automated metrics and vision-language models on identifying granular errors in detailed image descriptions and ranking paired descriptions against human judgments, using macro F1, pairwise accuracy, Spearman rank ρ, and Kendall's τ.

Use when planning an NSDI campaign across the spring and fall deadlines — sequencing abstract and paper gates, notification waits, one-shot revision windows, artifact evaluation, and the May symposium — so a networked-systems project always knows which of the two yearly gates it is really building toward.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.