Plugins

2 plugins

GKE Batch & Inference

For teams running batch/HPC and AI/ML inference workloads on GKE with specialized hardware.

2 skills · plugin

Deploy AI Inference on GKE

Deploy and optimize AI/ML inference workloads on GKE using GPUs, TPUs, and model servers.

3 skills · plugin

Results for “inference”

5 skills

Building Inferencesh Apps

Build and deploy applications on the inference.sh platform using Python or Node.js, with CLI scaffolding, local testing, and cloud deployment.

Tao Run Inference Service

Start, query, and stop a TAO inference microservice for a specific network architecture by delegating container execution to the appropriate platform skill.

More results

Tao Launch Workflow

Collects launch inputs and runs preflight checks before executing TAO workflows such as AutoML, training, evaluation, inference, export, TensorRT engine generation, or DEFT jobs on supported platforms.

Jetson Inference Mem Tune

Recommends an inference runtime and memory-related launch flags for LLM/VLM workloads on NVIDIA Jetson devices, based on a live memory audit snapshot.

Vss Deploy Video Embedding

Deploy and operate the VSS 3.2 GA RT-Embed Video Embedding microservice using Docker Compose, covering GPU prerequisites, REST API usage for file uploads, text/video embeddings, live RTSP streams, Redis/Kafka/OTel integration, and troubleshooting.