Results for “triton-inference-server”

Serve LLMs and VLMs on NVIDIA Jetson devices using vLLM or SGLang with optimized Docker containers and quantization presets.

Start, query, and stop a TAO inference microservice for a specific network architecture by delegating container execution to the appropriate platform skill.

Build video analytics pipelines using NVIDIA DeepStream SDK 9.0 with Python pyservicemaker API, including GStreamer-based video processing, TensorRT inference integration, object detection/tracking, and Kafka/message broker integration.

Recommends an inference runtime and memory-related launch flags for LLM/VLM workloads on NVIDIA Jetson devices, based on a live memory audit snapshot.

Build, deploy, evaluate, debug, and measure latency for a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection and VLM classification.

Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.

Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers.

Enable MIPI/GMSL camera sensors on a Jetson Thor or Orin custom carrier by rendering a kernel-DT overlay from the in-tree sensor DTSI.

Finetunes the NV-Generate-CTMR MR-brain diffusion UNet from user-supplied NIfTI training volumes using a wrapper that stages configs and delegates to upstream scripts.

Install, configure, and extend Pi, a terminal coding harness, with support for custom providers, models, extensions, skills, packages, themes, SDK integration, RPC mode, JSON event streams, and ecosystem packages for subagent delegation, MCP servers, interactive forms, and web access.

Moves issues and external PRs through a state machine of triage roles, categorizing, verifying, grilling, and writing agent-ready briefs.

Python SDK for inference.sh - run AI apps, build agents, and integrate with 150+ models. Package: inferencesh (pip install inferencesh). Supports sync/async, streaming, file uploads. Build agents with template or ad-hoc patterns, tool builder API, skills, and human approval. Use for: Python integration, AI apps, agent development, RAG pipelines, automation. Triggers: python sdk, inferencesh, pip install, python api, python client, async inference, python agent, tool builder python, programmatic ai, python integration, sdk python

Fine-tune and serve Physical Intelligence OpenPI models (pi0, pi0-fast, pi0.5) using JAX or PyTorch backends for robot policy inference across ALOHA, DROID, and LIBERO environments. Use when adapting pi0 models to custom datasets, converting JAX checkpoints to PyTorch, running policy inference servers, or debugging norm stats and GPU memory issues.

Conducts autonomous multi-source research using a planner/executor architecture and an MCP server, with tools for deep research, quick search, report writing, and source retrieval.

Manages vector embeddings with Pinecone for semantic search, recommendation, and RAG pipelines.

You are an expert in tRPC, the framework for building type-safe APIs without schemas or code generation. You help developers create full-stack TypeScript applications where the server defines procedures and the client calls them with full type inference — no REST routes, no GraphQL schemas, no OpenAPI specs, just TypeScript functions that are type-safe from database to UI.

Provides code examples and best practices for using Pinecone, a managed vector database for production RAG, recommendation, and semantic search applications.

在 AI 编程工作流中直接检索、分析和更新 Jira 工单。支持 MCP（推荐）和直接 REST API 两种方式。

Reduce OpenClaw token usage and API costs through smart model routing, heartbeat optimization, budget tracking, and native 2026.2.15 features (session pruning, bootstrap size limits, cache TTL alignment). Use when token costs are high, API rate limits are being hit, or hosting multiple agents at scale. The 4 executable scripts (context_optimizer, model_router, heartbeat_optimizer, token_tracker) are local-only — no network requests, no subprocess calls, no system modifications. Reference files (PROVIDERS.md, config-patches.json) document optional multi-provider strategies that require external API keys and network access if you choose to use them. See SECURITY.md for full breakdown.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.