Results for “nas-bench-nlp”

Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution on local Docker, Slurm HPC, or cloud platforms.

Build complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and agents with DSPy - Stanford NLP's framework for systematic LM programming

Build complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and agents with DSPy - Stanford NLP's framework for systematic LM programming

Build complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and agents with DSPy - Stanford NLP's framework for systematic LM programming

Load pre-trained models from Hugging Face Hub, run pipeline inference, generate text, and fine-tune models on NLP, vision, audio, and multimodal tasks using the Transformers library.

China LLM Gateway - Unified interface for Chinese LLMs including Qwen, DeepSeek, GLM, Baichuan. OpenAI compatible, one API Key for all models. Use when: the user needs model routing, provider setup, or Chinese LLM access guidance.

Builds NLP pipelines with Haystack for document search, QA, and LLM-powered applications.

Benchmark LLM performance across tasks — latency, quality, cost comparison.

Fast tokenization for NLP using Rust-based tokenizers supporting BPE, WordPiece, and Unigram algorithms, with training, alignment tracking, and padding/truncation.

Audits natural-language programming artifacts such as SKILL.md, AGENTS.md, CLAUDE.md, slash commands, plugin manifests, hooks, rules, and prompt files. Use when reviewing AI-agent repositories, checking manifest-vs-disk consistency, scoring skill or agent quality, adding NL artifact CI gates, or diagnosing vocabulary and version drift across Claude Code, Codex, Cursor, Gemini, and Antigravity-style projects.

Build complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and agents with DSPy - Stanford NLP's framework for systematic LM programming

Measures inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Deploy and serve LLMs with high throughput using vLLM's PagedAttention and continuous batching. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism for production inference.

---

Official Notion Model Context Protocol Server for workspace interaction.

Import PyTorch, ONNX, or Keras 3 / TensorFlow 2.16+ deep learning models into MATLAB as dlnetwork objects. Use when importing .pt2 exported programs, traced .pt files, .onnx models, or Keras 3 models via matlabsaver. Covers importNetworkFromPyTorch, importNetworkFromONNX, importNetworkFromKeras, importNetworkFromTensorFlow, torch.export.export, PyTorchInputSizes, InputDataFormats, matlabsaver, tf_keras downgrade, numeric validation against PyTorch or ONNX Runtime, and placeholder/custom layer implementation. Applies when user mentions any of these functions, file formats, or encounters import errors, unsupported operator warnings, 0 learnables, or uninitialized networks.