agentskillexchange Route agent LLM traffic through Shepherd Model Gateway
Use Shepherd Model Gateway to route OpenAI, Anthropic, Responses API, embeddings, and MCP tool traffic across self-hosted and cloud model backends with cache-aware policies and observability.
Prerequisites
SMG binary, Docker image, Python package, or Rust install; one or more model worker endpoints; agent runtime configured to call the gateway; optional Prometheus/OpenTelemetry stack
Installation
Use the upstream install or setup path that matches your environment:
- docker pull lightseekorg/smg:latest
- pip install smg
- cargo install smg
Requirements and caveats from upstream:
- | High Availability | Mesh networking with SWIM protocol for multi-node deployments |
Basic usage or getting-started notes:
Install — pick your preferred method:
Run — point SMG at your inference workers:
smg launch --worker-urls http://localhost:8000
Extracted from upstream docs: https://raw.githubusercontent.com/lightseekorg/smg/HEAD/README.md