Results for “evaluation”

Build automated evaluation pipelines for Python LLM applications using real LLM calls and structured test datasets.

Diagnose and improve MSBuild project evaluation performance by analyzing phases, glob patterns, import chains, and property functions.

Manage Arize datasets and examples using the ax CLI: create, list, get, export, and append datasets for evaluation and experimentation.

Create, run, and analyze evaluation suites for Microsoft 365 Copilot declarative agents using the @microsoft/m365-copilot-eval CLI.

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware, with backend selection between vLLM, Transformers, and accelerate.

Build and run evaluators for AI/LLM applications using Phoenix, covering error analysis, custom evaluators, experiments, and production monitoring.

Fine-tune models on Azure AI Foundry using SFT, DPO, or RFT, covering dataset preparation, training job submission, deployment, and evaluation.

Helps software engineers with discrete job search tasks: job description analysis, CV tailoring, cover letter writing, offer evaluation, and follow-up emails.

Implement iterative evaluation and refinement loops for AI agent outputs, using self-critique, evaluator-optimizer patterns, and rubric-based scoring to improve quality.

Validate enterprise workloads against Google Cloud best practices using public client libraries and the REST API to manage evaluations, rules, scanned resources, and validation results.

Build AI applications on Microsoft Foundry using the Azure AI Projects Python SDK, including agent creation, tool integration, thread management, and evaluation.

Provides canonical MSBuild property definition patterns including conditional defaults, composition, path normalization, target framework detection, and evaluation order for diagnosing and fixing property issues in .props and .csproj files.

Improve CLI build performance by enabling MSBuild Server for persistent caching of evaluation results across sequential builds.

Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.

Manage NVIDIA Brev GPU instances for TAO training, evaluation, and inference using the Brev CLI and Docker.

Evaluates RAG pipelines using a filesystem-based benchmark with corpus/ and train.json, running evaluate_rag.py to tune retrieval and generation flags and interpret RAGAS metrics.

Plan, debug, tune, evaluate, export, or deploy public Nemotron embedding and reranking retrieval recipes using the current checkout.

Train, evaluate, distill, quantize, export, and run inference for RT-DETR object detection models using NVIDIA TAO.

Build AI applications using the Azure AI Projects SDK for TypeScript, managing agents, connections, deployments, datasets, indexes, and evaluations.

Fine-tune a TAO model with standard supervised training, evaluation, and export, with AutoML bypass and platform-specific credential intake.

Manage Azure AI Foundry projects with .NET SDK: create and run agents, manage connections, datasets, deployments, evaluations, and indexes.

Deploy, evaluate, fine-tune, and manage Microsoft Foundry agents end-to-end using Azure Developer CLI and MCP tools.

Manage Arize AI Observability and Evaluation resources on Azure using the .NET SDK. Create, read, update, and delete Arize organizations via Azure Marketplace.

Creates and manages annotation configs and annotation queues on Arize, and applies human annotations to project spans via the Python SDK.

Curates clinical-specialty term lists, generates IPA-tagged synthetic audio via TTS, and produces NeMo-format manifests for ASR benchmark evaluation.

Submit and manage TAO training, evaluation, and inference jobs on SLURM GPU clusters over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed storage.

Collects launch inputs and runs preflight checks before executing TAO workflows such as AutoML, training, evaluation, inference, export, TensorRT engine generation, or DEFT jobs on supported platforms.

Guides the creation of high-quality MCP servers that enable LLMs to interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation.

Bootstraps a clinical ASR evaluation environment by verifying NVIDIA_API_KEY, installing Python dependencies, and running a smoke test against hosted TTS/ASR services.

Plan, configure, and chain Nemotron model customization steps into single-step or multi-step pipelines for curation, translation, fine-tuning, RL alignment, benchmarking, checkpoint conversion, optimization, and evaluation.