Results for “inference-latency”

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.

Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.

Slow down interpretation from observation to action. Use when students or adults need to examine assumptions in conflict, dialogue, or inquiry.

Trains, evaluates, exports, and runs inference for FastFoundationStereo (FFS) stereo depth estimation models, a distilled variant of FoundationStereo with lower latency.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques for 1.5-3.6× speedup without quality loss.

The layers trade timeliness against definitiveness.

Authenticates and connects to Google Cloud Agent Platform for inference with Gemini and third-party OpenMaaS models (Llama, DeepSeek, Qwen). Generates code for multiple SDKs, configures endpoints, and troubleshoots common errors.

Orchestrate token-heavy research, coding, and testing by delegating bounded tasks to cheaper subagents while reserving Claude Fable for architecture, synthesis, and final review.

Design privacy-aware observability for AI agents using traces, spans, structured events, metrics, cost attribution, dashboards, alerts, and investigation workflows. Use when instrumenting an agent, debugging intermittent tool or model failures, defining service-level objectives, analyzing latency or spend, auditing agent decisions, or preparing production monitoring.

Expert en inference llama.cpp avancé (GGUF, quantization, local models, HTTP server, hardware)

Econometrics skill for panel data models. Activates when the user asks about: "panel data", "fixed effects", "random effects", "Hausman test", "within estimator", "between estimator", "two-way fixed effects", "clustered standard errors panel", "FE model", "RE model", "pooled OLS", "unobserved heterogeneity", "panel regression", "first difference estimator", "entity fixed effects", "time fixed effects", "面板数据", "固定效应", "随机效应", "豪斯曼检验", "双向固定效应", "面板回归", "个体效应", "时间效应", "一阶差分"

`analysis-agent`/`task-agent`/`review-agent`: use when latency, throughput, bundle, memory, CPU, query, rendering, or resource cost needs a budget; skip without performance risk.

Build AI applications with the inference.sh Python SDK: run apps, build agents, and integrate with 250+ models using sync/async, streaming, file uploads, and a tool builder API.

After any AI-generated explanation, require the learner to identify one place it could be wrong, one thing to check, and one source to consult. Builds epistemic vigilance — treats AI output as a claim to evaluate, not truth to absorb.

Econometrics skill for panel data models. Activates when the user asks about: "panel data", "fixed effects", "random effects", "Hausman test", "within estimator", "between estimator", "two-way fixed effects", "clustered standard errors panel", "FE model", "RE model", "pooled OLS", "unobserved heterogeneity", "panel regression", "first difference estimator", "entity fixed effects", "time fixed effects", "面板数据", "固定效应", "随机效应", "豪斯曼检验", "双向固定效应", "面板回归", "个体效应", "时间效应", "一阶差分"

選択肢(A/B/C)の深掘り比較→淘汰→推奨で判断負担を下げ判断の質を上げるスキル。5段階（L1固定3点/L1.5案拡張Diverge・自動/L2評価軸マトリクス/L3複数LLM弁証論/L4過去判断照合）。 「比較して」「深掘りして」「メリデメ教えて」「お勧めは？」「徹底的に」「過去の判断と照合」「前にどう決めたっけ」「/sentaku」等で発火。teian(浅)の深掘り要求を受け取り、brainstorming(深:設計全体)と棲み分け。

Instrument a shipped product's AI agents with tracing and observability so you can see what they did, why outputs happened, and what each run cost. Plain-language primer plus free-tier-first backend selection (Langfuse, Phoenix, LangSmith, Braintrust) and OpenTelemetry/OpenInference instrumentation. Load when the user asks to add observability, add tracing, instrument my agents, see what my agent is doing in production, set up Langfuse or Phoenix or LangSmith, debug why my agent gave a bad answer, or track LLM cost per request. Also fires when agent-system-architecture or setup-evaluation requires an observability plan for an agent-chain product. NOT for tracing the coding agent itself — that is run-trace. Precondition for runtime-learning-loop.

Debug and optimize AI agents by analyzing reasoning traces, context degradation, tool confusion, instruction drift, repeated task failures, and performance regressions.

Causal inference framework for answering "does X cause Y?" beyond correlation. DoWhy (Microsoft Research) provides the identify-estimate-refute loop: define a causal graph (DAG), identify the causal effect using backdoor/frontdoor/instrumental variable criteria, estimate treatment effects with multiple estimators, and validate results with automated refutation tests. Use when: distinguishing causation from correlation, estimating treatment effects (ATE, ATT, CATE), designing and analyzing A/B tests with confounders, using instrumental variables, performing counterfactual reasoning ("what would have happened if..."), validating causal claims with sensitivity analysis, working with observational data where randomization is impossible, or any analysis where the question is "what is the CAUSAL effect of X on Y" rather than just "how do X and Y relate?"

Evaluates how well internal model representations (hidden states) predict token-level information importance in summarization tasks, using NDCG@10 and Spearman's rank correlation.

Capture confidence ratings before and after a learning attempt to identify overconfidence and underconfidence patterns. Use when a student wants to understand how well they actually know something versus how well they think they know it.

Diagnose and mitigate context degradation patterns including lost-in-middle failures, context poisoning, distraction, confusion, and clash in AI agent systems.

Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments. Use when setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, or profiling inference latency on cluster or local GPU machines.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

Expert guidance for Groq, the LLM inference platform that provides the fastest token generation speeds available, powered by custom LPU (Language Processing Unit) hardware. Helps developers integrate Groq's API for real-time AI applications where latency matters — chatbots, code completion, and streaming responses.

Plans and coordinates the deployment of a model to Amazon SageMaker AI, selecting the appropriate pathway (real-time, serverless, async, batch, or Bedrock CMI) based on model type, traffic, latency, and cost constraints.