Results for “l-eval”

Benchmarks automatic speech translation and recognition on English-French and English-Romanian datasets, reporting BLEU and WER on tokenized outputs.

Provides guidance for automatically evolving and optimizing AI agents across any domain using LLM-driven evolution algorithms. Use when building self-improving agents, optimizing agent prompts and skills against benchmarks, or implementing automated agent evaluation loops.

Train, evaluate, export, and run inference for Masked Auto-Encoder (MAE) models for self-supervised pretraining and fine-tuning of visual representations.

Benchmarks large audio-language models against adversarial audio attacks using the CAA dataset, computing WER, ROUGE-L, cosine similarity, and coherence scores to assess robustness in conversational settings.

Guides the creation of high-quality MCP servers that enable LLMs to interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation.

Fine-tune, evaluate, run inference, and export Cosmos-Embed1 video-text embedding models for tasks like text-to-video retrieval and semantic deduplication.

Validates the Emotional Attitude Score (EAS) metric by measuring its consistency with human judgment on word-level sentiment polarity, using the AmbGIMT dataset and pairwise score comparisons.

Model software around the business domain using bounded contexts, aggregates, and ubiquitous language, with scoring and diagnostic tools for evaluating domain model quality.

Guides the creation of high-quality MCP servers that let LLMs interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation.

Evaluates Power BI data model architecture, relationships, storage modes, and performance to identify optimization opportunities and ensure adherence to best practices.

Discovers and evaluates community skills from skills.sh and awesome-openclaw-skills when local skills don't cover a need, scoring credibility and installing top recommendations with permission.

Route LangSmith work into one workflow packet before touching SDK code. Use when the user needs LangSmith tracing, offline evals, annotation/review queues, prompt-registry decisions, audit/gap review, or cross-service trace propagation for an LLM app or agent workflow. Choose one packet: trace-debug, eval, review, prompt-registry, propagation, or audit. Triggers on: LangSmith, LangChain tracing, `@traceable` / `traceable`, `wrap_openai` / `wrapOpenAI`, datasets, experiments, annotation queues, feedback criteria, Prompt Hub, run trees, trace IDs, or production confidence for an AI feature. Not for generic SLO/alert design, non-LangSmith deployment orchestration, or runtime guardrails outside LangSmith.

Trains, evaluates, and runs inference for Mask Auto-Label (MAL) weakly-supervised segmentation models using ViT-MAE backbones with minimal point or box annotations.

Build, deploy, evaluate, debug, and measure latency for a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection and VLM classification.

The learner teaches the concept to the AI, which plays a curious novice peer and identifies gaps through authentic questions. Use when the learner wants to test their understanding — teaching forces a different kind of organisation than studying.

Measures inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations.

Train, evaluate, export, and run inference on TAO action-recognition models for classifying temporal actions in video clips using RGB, optical flow, or joint input.

Discover, evaluate, install, and wrap external agent skills from ClawHub registry, skillflag-compliant CLI tools, and GitHub repos. Use when asked to find new capabilities, install a skill, browse what's available, or package a local skill for sharing.

Agent profile for evaluate and implement no-code/low-code workflows, forms, automations, Airtable/Sheets/Zapier-like patterns. Use when Codex needs a specialist agent perspective for planning, implementation, review, debugging, validation, or handoff in this domain.

Trains, evaluates, and runs inference for BEVFusion multi-sensor 3D object detection models that fuse LiDAR and camera data in bird's-eye-view space for autonomous driving.

Evaluates Base ERC-20 tokens by contract address, returning a 0-100 score across five criteria, launch protocol classification, security flags, and a narrative interpretation.

Guides the creation of high-quality MCP servers that enable LLMs to interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation across multiple programming languages.

Generate a Startup Canvas combining Product Strategy (9 sections) and Business Model (costs + revenue) for a new product. An alternative to BMC and Lean Canvas that separates strategy from business model. Use when launching a new product or evaluating a startup concept.

Sets up Google's agents-cli toolkit and guides through the full agent development lifecycle: scaffold, build, evaluate, deploy, publish, and monitor on Gemini Enterprise Agent Platform.

Run a PDCA quality audit on LLM/AI features: traces, prompts, costs, evals, grounding, hallucination. Use for "audit LLM quality", "check Langfuse", "audit prompts", "check AI quality", "audit AI costs", "check traces". Jailbreak/OWASP LLM → audit-llm-security. Token caps → plan-llm-cost-guardrails.

After any AI-generated explanation, require the learner to identify one place it could be wrong, one thing to check, and one source to consult. Builds epistemic vigilance — treats AI output as a claim to evaluate, not truth to absorb.

Benchmarks LLM agents fine-tuned with the Agent Data Protocol across software engineering, web browsing, OS/database tool use, and reasoning tasks, reporting unit test pass rates and task success rates.

Fine-tunes and evaluates OpenVLA-OFT and OpenVLA-OFT+ policies for robot action generation with continuous action heads, LoRA adaptation, and FiLM conditioning on LIBERO simulation and ALOHA real-world setups.

ML / research person pack for Content Factory. Use when the user says person ml, @person-ml, ML person, research scientist person, or LLM researcher person. Auto-loads ml-research-engineer, llm-researcher, ai-research-scientist plus experiment-tracking, evaluation, cf-llm-model-usage.

Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.

Automates the full DEFT AOI improvement loop for NVIDIA TAO VisualChangeNet / ChangeNet PCB inspection models, including baseline evaluation, RCA, synthetic defect generation, data mining, retraining, and deployment gating until KPI targets are met.

Security audit and vulnerability scanner for AI agent skills before installation. Use when: (1) evaluating a skill from an untrusted source, (2) auditing a skill directory or git repo URL for malicious code, (3) pre-install security gate for Claude Code plugins, OpenClaw skills, or Codex skills, (4) scanning Python scripts for dangerous patterns like os.system, eval, subprocess, network exfiltration, (5) detecting prompt injection in SKILL.md files, (6) checking dependency supply chain risks, (7) verifying file system access stays within skill boundaries. Triggers: "audit this skill", "is this skill safe", "scan skill for security", "check skill before install", "skill security check", "skill vulnerability scan".

Quantifies how sensitive a language model benchmark's reliability and ranking stability are to specific design choices, such as the selection of scenarios, subscenarios, examples, and few-shot prompts. Use when the user has predictions and gold and needs to compute DIoR.

Run Karpathy-style autonomous ML search on a real training repo: choose the right mode (setup, program.md, bounded loop, results interpretation, or constrained-hardware adaptation), preserve the immutable prepare.py / 300-second / val_bpb contract, and route prompt/skill eval work away to LangSmith, Promptfoo, Braintrust, or skill-autoresearch.

Conducts Data Protection Impact Assessments for AI and ML systems per EDPB Guidelines 04/2025 on AI processing. Covers training data lawfulness evaluation, model risk assessment, automated decision triggers, and AI-specific DPIA methodology. Keywords: AI DPIA, machine learning impact assessment, EDPB AI guidelines, model risk, training data.

Build type-safe AI agents and graph-based workflows with PydanticAI and PydanticGraph. Agent creation, function tools, capabilities, dependency injection, structured output, streaming, multi-agent patterns, testing, evals, and graph state machines. Use whenever you are building agents, tool-using LLM workflows, or graph-based state machines in Python.