Data & Analytics

qhjqhj00 Skill

Evaluates a Text-to-SQL framework's ability to generate correct and efficient SQL queries from natural language questions across complex, cross-domain databases. It probes schema filtering, multi-generator candidate creation, and selection robustness. Use when the user wants to benchmark on BIRD, Spider, or asks about evaluating this task. Reports Execution Accuracy (EX).

qhjqhj00 Skill

Evaluates the cross-framework portability and reusability of declarative agent specifications by executing identical agentic workflows across four different runtime frameworks (AutoGen, CrewAI, LangGraph, WayFlow) on three distinct task benchmarks. Use when the user wants to benchmark on SimpleQA Verified, BIRD-SQL, $\tau^{2}$-Bench, or asks about evaluating this task. Reports F1 score, EX%, Passˆk.

qhjqhj00 Skill

Evaluates the capability of large language models to generate correct SQL queries from natural language questions. It specifically probes how annotation noise and errors in benchmark datasets affect model performance and reliability. Use when the user wants to benchmark on BIRD-Bench, or asks about evaluating this task. Reports accuracy.

qhjqhj00 Skill

This benchmark evaluates the robustness and accuracy of Text-to-SQL systems when translating natural language questions into SQL queries across different database schema designs. It probes how data model complexity, training data size, and language model scale impact execution accuracy on real-world user queries. Use when the user wants to benchmark on FootballDB, or asks about evaluating this task. Reports exact execution matching (EX).

qhjqhj00 Skill

Evaluates the zero-shot and few-shot generalization capability of text-to-SQL parsers on realistic, industrial-style database schemas with obscure column names and unrestricted natural language questions. It probes the model's ability to perform schema linking, constraint parsing, and SQL generation without extensive domain-specific training data. Use when the user wants to benchmark on KaggleDBQA, or asks about evaluating this task. Reports exact-match accuracy.

qhjqhj00 Skill

Evaluates language models' ability to perform real-world enterprise text-to-SQL workflows. It probes agentic reasoning, multi-step data transformation, database schema navigation, and SQL dialect adaptation across complex, long-context tasks. Use when the user wants to benchmark on Spider 2.0, Spider 2.0-lite, Spider 2.0-snow, or asks about evaluating this task. Reports Success Rate (SR).

qhjqhj00 Skill

This benchmark evaluates the robustness of text-to-SQL models when natural language questions contain real-world synonyms replacing schema-related terms. It probes whether models rely on rigid lexical matching or can generalize to paraphrased queries while preserving the underlying database schema and target SQL query. Use when the user wants to benchmark on Spider, Spider-Syn, or asks about evaluating this task. Reports exact matching accuracy.

qhjqhj00 Skill

Evaluates an LLM's ability to generate correct SQL queries for transforming building energy data schemas. It measures how well different prompt strategies and iterative optimization handle complex schema mappings, pivoting, and aggregation in real-world smart building datasets. Use when the user wants to benchmark on Building Energy Data Transformation Benchmark, or asks about evaluating this task. Reports Execution Accuracy.

qhjqhj00 Skill

This benchmark evaluates an LLM's ability to debug and fix real-world, user-reported SQL issues. It probes the model's capacity to identify logical, syntactic, and semantic flaws in existing queries and generate correct, functional SQL replacements across different database dialects and complexity levels. Use when the user wants to benchmark on BIRD-CRITIC, or asks about evaluating this task. Reports Success Rate (SR).

qhjqhj00 Skill

Evaluates automated segmentation and diagnostic classification capabilities on cardiac magnetic resonance imaging (CMR) sequences. Probes the model's ability to accurately delineate cardiac structures across multiple anatomical views and classify cardiovascular diseases with clinical-grade metrics. Use when the user wants to benchmark on BAAI Cardiac CMR Cohort, or asks about evaluating this task. Reports DSC, AUC.

qhjqhj00 Skill

Evaluates a model's ability to perform dynamic, multi-turn Text-to-SQL interactions that support full CRUD operations. It probes contextual reasoning, adaptability to evolving user requests, and error recovery within stateful database dialogues. Use when the user wants to benchmark on DySQL-Bench, or asks about evaluating this task. Reports state-equivalence accuracy.

qhjqhj00 Skill

Evaluates a system's ability to translate natural language clinical questions into executable SQL and retrieve accurate results from a specialized electronic health record database. It probes complex temporal reasoning, clinical constraint handling, and semantic equivalence in text-to-SQL generation. Use when the user wants to benchmark on EHRSQL 2024, or asks about evaluating this task. Reports execution accuracy.

qhjqhj00 Skill

Evaluates a neural semantic parser's ability to map natural language utterances directly to executable SQL queries. It probes compositional generalization and schema grounding by measuring whether predicted queries return the exact same results as gold queries on a target database. Use when the user wants to benchmark on GEO880, ATIS, or asks about evaluating this task. Reports denotation accuracy.

qhjqhj00 Skill

Evaluates the effectiveness and efficiency of an LLM-based index recommendation system in selecting database indexes for given SQL workloads under varying storage constraints and schema generalization settings. Use when the user wants to benchmark on TPC-H, JOB, TPC-DS, SSAG, AMPS, or asks about evaluating this task. Reports Relative Workload Cost Reduction.

qhjqhj00 Skill

Evaluates a model's ability to translate natural language questions into correct SQL queries for a complex, real-world industrial database. It also probes schema-linking precision by measuring how accurately the model identifies the required tables from the schema. Use when the user wants to benchmark on Industrial Energy Database Benchmark, or asks about evaluating this task. Reports Accuracy.

qhjqhj00 Skill

Evaluates a unified dialogue foundation model across representation, knowledge distillation, and generation capabilities on diverse dialogue-oriented tasks. It probes the model's ability to perform intent detection, slot filling, semantic parsing, dialogue state tracking, text-to-SQL, and end-to-end task-oriented dialogue generation. Use when the user wants to benchmark on DialoGLUE, MULTIWOZ2.0, MULTIWOZ2.2, Spider, CoSQL, CLINC150, BANKING77, HWU64, RESTAURANT8K, DSTC8, TOP, PERSONALCHAT, COQA, SAMSUM, CANARD, or asks about evaluating this task. Reports exact match (EM).

qhjqhj00 Skill

Evaluates the text-to-SQL and dialog state tracking capabilities of language models by measuring how accurately they generate syntactically and semantically valid SQL queries from natural language questions, with and without constrained auto-regressive decoding. Use when the user wants to benchmark on Spider, CoSQL, or asks about evaluating this task. Reports exact-set-match accuracy.

qhjqhj00 Skill

Evaluates an LLM's ability to translate SQL queries across different database schemas while preserving structural integrity and semantic meaning. It measures mapping success, structural fidelity, execution validity, and the semantic alignment between generated SQL and natural language questions. Use when the user wants to benchmark on BIRD, SPIDER, or asks about evaluating this task. Reports Structural Alignment.

qhjqhj00 Skill

Evaluates the capability of large language models (ChatGPT and GPT-4) and domain-specific models to solve a variety of financial text analytics tasks. It probes performance across sentiment analysis, classification, information extraction, and question answering, measuring how well models handle domain-specific knowledge and structured prediction. Use when the user wants to benchmark on Financial NLP Tasks (Sentiment, Classification, NER, RE, QA), or asks about evaluating this task. Reports accuracy.

qhjqhj00 Skill

Evaluates a model's ability to detect unanswerable Text-to-SQL queries by analyzing intermediate hidden activations, aiming to prevent hallucinated SQL generation and unsafe execution. It probes whether the system can reliably distinguish between answerable and unanswerable prompts across diverse domains and linguistic ambiguities. Use when the user wants to benchmark on TriageSQL, AMBROSIA, SQuAD 2.0, MD-Enterprise, or asks about evaluating this task. Reports F1.

qhjqhj00 Skill

Evaluates the task performance, latency, and hardware utilization of quantized LLMs and LMMs on real mobile devices across standard NLP, multi-modal, and trust & safety benchmarks. Use when the user wants to benchmark on Databricks, HotpotQA, sql-create-context, CNN, XSum, VQA-v2, GQA, VisWiz, TextVQA, SQA, AlpacaEval, MT-Bench, MMLU, GSM8K, TruthQA, BBQ, SC-101, Adv-Inst, DNA, Priv-Lk, or asks about evaluating this task. Reports Accuracy.

qhjqhj00 Skill

Evaluates the quality of synthetically generated Text-to-SQL data by measuring question-SQL alignment and business realism in a sales analytics domain. Use when the user wants to benchmark on Salesforce Sales Analytics Database, or asks about evaluating this task. Reports Question-SQL Alignment (%).

qhjqhj00 Skill

Evaluates text-to-SQL generation capabilities across cross-domain parsing, knowledge-intensive reasoning, and enterprise-level SQL workflows. It also assesses the semantic validity, execution correctness, and diversity of synthetically generated training data. Use when the user wants to benchmark on Spider, BIRD, Spider2.0, EHRSQL, ScienceBenchmark, Spider-Syn, Spider-Realistic, Spider-DK, or asks about evaluating this task. Reports test-suite accuracy (TS), execution accuracy (EX).

qhjqhj00 Skill

Evaluates how synthetic or human-generated column descriptions impact LLM performance on text-to-SQL tasks, and assesses the quality of LLM-generated descriptions across varying semantic difficulty levels. Use when the user wants to benchmark on BIRD-Bench, or asks about evaluating this task. Reports Mean quality scores.

qhjqhj00 Skill

Evaluates open-domain table retrieval and end-to-end question answering over complex table reasoning tasks. It probes a model's ability to retrieve relevant table segments from a corpus and then answer questions using either direct reading or SQL generation. Use when the user wants to benchmark on Open-WikiTable, or asks about evaluating this task. Reports Top-k table retrieval accuracy.

qhjqhj00 Skill

Evaluates an end-to-end navigation model's ability to predict robot dynamics and successfully navigate through structured and cluttered warehouse environments. It probes both open-loop trajectory and speed prediction accuracy, as well as closed-loop mission success, navigation efficiency, and motion smoothness in seen and out-of-distribution settings. Use when the user wants to benchmark on X-Mobility Warehouse Dataset, or asks about evaluating this task. Reports mission success rate (SR).

qhjqhj00 Skill

Evaluates AI agents' ability to execute end-to-end investment banking workflows, requiring multi-file deliverable generation (Excel, PowerPoint, reports) using specialized financial tools and data sources. It probes financial judgment, cross-artifact consistency, tool-use fidelity, and professional presentation standards under realistic constraints. Use when the user wants to benchmark on BankerToolBench, or asks about evaluating this task. Reports rubric score.

qhjqhj00 Skill

Evaluates long-video understanding and agentic retrieval capabilities by testing models on temporal grounding, summarization, reasoning, and event causality across ultra-long video streams. Use when the user wants to benchmark on LVBench, VideoMME-Long, Ava-100, or asks about evaluating this task. Reports accuracy.

qhjqhj00 Skill

Evaluates an LLM's ability to generate structurally complex SQL queries for real-world decision-making workloads. It probes the model's capacity to handle deep nesting, multiple joins, diverse column references, and complex filtering conditions compared to simpler benchmarks. Use when the user wants to benchmark on TPC-DS, or asks about evaluating this task. Reports structural_similarity.

qhjqhj00 Skill

Evaluates a model's capability to generate fluent natural language descriptions from SQL queries (SQL-to-text) and measures how well the generated text can augment training data for Text-to-SQL parsers. Use when the user wants to benchmark on WikiSQL, Spider, or asks about evaluating this task. Reports BLEU-4.

qhjqhj00 Skill

Evaluates text-to-query systems across relational, document, and graph database models using four query languages (SQL, MQL, Cypher, SPARQL). It probes the ability of models to translate natural language medical questions into correct, executable database queries using standardized SNOMED-CT aligned synthetic patient data. Use when the user wants to benchmark on SM3-Text-to-Query, or asks about evaluating this task. Reports correctness.

qhjqhj00 Skill

Evaluates the ability of LLMs to generate executable structured queries (SQL, SPARQL) from natural language questions over tables, knowledge graphs, and temporal knowledge graphs. It specifically probes the model's capacity for iterative self-correction when initial query generation fails, measuring both raw generation accuracy and error-resilience during multi-round inference. Use when the user wants to benchmark on WikiSQL, WTQ, MetaQA, WebQSP, CronQuestions, TabFact, or asks about evaluating this task. Reports Denotation accuracy.

qhjqhj00 Skill

Evaluates large language models' ability to generate correct SQL or SPARQL queries for natural language questions over an enterprise insurance database. It measures how well zero-shot prompting with raw schema versus knowledge graph augmentation improves factual grounding and query execution accuracy. Use when the user wants to benchmark on Enterprise SQL & KG QA Benchmark, or asks about evaluating this task. Reports execution accuracy.

qhjqhj00 Skill

Evaluates the ability of program-by-example (PBE) systems to synthesize correct SQL queries from example input/output tables. It probes query generation accuracy, synthesis speed, and scalability to larger database schemas. Use when the user wants to benchmark on ase13, so-top, so-dev, so-rec, kaggle, or asks about evaluating this task. Reports solve_rate.

qhjqhj00 Skill

Evaluates a model's capacity for metric spatial reasoning, object enumeration, relative spatial comparisons, and topological/directional relationship understanding within real-world warehouse environments. Use when the user wants to benchmark on PhysicalAI-Spatial-Intelligence-Warehouse, or asks about evaluating this task. Reports normalized exact-match accuracy.

qhjqhj00 Skill

Evaluates large language models' ability to perform complex, multi-step reasoning and data dependency tracking by executing SQL queries over synthetic, arbitrarily long tables. It probes contextual reasoning, long-context understanding, and structured data manipulation capabilities beyond traditional benchmarks. Use when the user wants to benchmark on S3Eval, or asks about evaluating this task. Reports SQL execution performance.