Results for “eval”

Designs and optimizes prompts for LLM-powered applications, covering system prompt architecture, context management, output formatting, and evaluation.

专业评估故事大纲，从市场潜力、创新属性、内容亮点维度判断评分。适用于故事大纲质量评估、IP改编潜力判断、项目立项决策

Scaffolds eval.yaml test files for agent skills in the dotnet/skills repository, including scenario writing, assertion definition, and rubric creation.

Apply foundational design principles—affordances, signifiers, constraints, mappings, and feedback—to evaluate and improve product usability, bridging the gulfs of execution and evaluation.

Scaffolds a coder-eval task suite in a repository by scanning for testable components, writing one real task, and validating it.

Process in-memory tabular data with a fast, expression-based DataFrame library that supports lazy evaluation, parallel execution, and Apache Arrow semantics.

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

Design multi-agent system architectures, generate tool schemas for Anthropic and OpenAI formats, and evaluate execution logs for cost, latency, and failure bottlenecks.

Rates an agent's own output on five axes — accuracy, completeness, clarity, actionability, conciseness — producing a structured scorecard with evidence and improvement suggestions.

Evaluates whether a lawsuit is worth pursuing, explains the litigation process from filing to resolution, and guides case preparation, settlement negotiations, and small claims alternatives.

Interrogates research plans before evals or published claims, forcing explicit null hypotheses, confound checks, and data verification across four review waves.

Analyzes evaluation results by requiring a stated hypothesis before examining data, then compares expectations to actual result files to prevent post-hoc rationalization.

Evaluates geometric consistency in text-to-video generation by measuring structural and motion coherence across camera trajectories, detecting deformation and occlusion artifacts in static scenes.

Evaluate retrieval quality from the local RAG index

LLM observability with Langfuse — tracing, evals, prompt management, cost tracking

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when "langfuse, llm observability, llm tracing, prompt management, llm evaluation, monitor llm, debug llm, langfuse, observability, tracing, llm-monitoring, evaluation, prompt-management, debugging, analytics" mentioned.

Deploy, evaluate, fine-tune, and manage Microsoft Foundry agents end-to-end using Azure Developer CLI and MCP tools.

Fine-tune models on Azure AI Foundry using SFT, DPO, or RFT, covering dataset preparation, training job submission, deployment, and evaluation.

Helps software engineers with discrete job search tasks: job description analysis, CV tailoring, cover letter writing, offer evaluation, and follow-up emails.

Process data with high-performance DataFrames using Polars' expression-based API, lazy evaluation, and parallel execution for ETL, analytics, and pandas migration.

Evaluates text-to-speech systems by measuring distributional distance between synthetic and real speech across five factors, producing a scalar score without subjective MOS ratings.

Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator. Use when the user runs /ar:setup or asks to start optimizing a file with the autoresearch loop.

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

Guides AI agents through engineering workflows with eval-first execution, task decomposition, cost-aware model routing, and review focus for generated code.

Provides structured guidance for building production-grade AI agent systems through context engineering, covering fundamentals, architectural patterns, operational excellence, and evaluation.

Simulates meiotic recombination to produce offspring genomes from parent pairs, modeling Mendelian segregation, de novo mutation, sex determination, trait inference, and clinical evaluation against a disease registry.

Guides writing Korean government funding program business plans (사업계획서) from an evaluator's perspective, covering structure, scoring emphasis, and writing principles.

Process in-memory datasets with Polars' expression API, lazy evaluation, and parallel execution, including pandas migration patterns and I/O for CSV, Parquet, and JSON.

Computes the HARE Score, an entity- and relation-centric metric for evaluating machine-generated histopathology reports against ground truth, using GatorTronS+SapBERT embeddings and relation F1.

Benchmarks LLM bias by scoring responses to automatically generated open-ended questions across sensitive attributes, producing a composite fitness score from 0 to 5.

Evaluate the codebase and suggest ideas to explore with $feature-interview

Every metric encodes an opinion about which mistake hurts.

Convert delivery findings into skill, eval, workflow, and documentation improvements.

Validate enterprise workloads against Google Cloud best practices using public client libraries and the REST API to manage evaluations, rules, scanned resources, and validation results.

Build AI applications on Microsoft Foundry using the Azure AI Projects Python SDK, including agent creation, tool integration, thread management, and evaluation.