# Onnx Runtime

> ONNX Runtime — cross-platform ML inference optimizer. Convert PyTorch, TensorFlow, scikit-learn models to ONNX. GPU, CPU, and mobile acceleration. Quantization, graph optimization, and custom ops.

- Skill: `mkurman/onnx-runtime` (Agent Skill)
- Install (CLI): `npx skillmds@latest add mkurman/onnx-runtime`
- Raw SKILL.md: https://api.skillmd.com/api/skills/mkurman/onnx-runtime/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: mkurman (https://skillmd.com/u/mkurman)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/mkurman/onnx-runtime

---

## Overview

ONNX Runtime is a cross-platform ML inference engine that runs models in the ONNX format. Supports CPU, GPU (CUDA, DirectML), and mobile inference with graph optimizations, quantization, and custom operators.

## Installation

```bash
uv pip install onnxruntime  # CPU
uv pip install onnxruntime-gpu  # CUDA

# Convert first: from transformers, PyTorch, etc.
```

## Basic Inference

```python
import onnxruntime as ort
import numpy as np

session = ort.InferenceSession("model.onnx")
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name

result = session.run([output_name], {input_name: np.random.randn(1, 3, 224, 224).astype(np.float32)})
print(result[0].shape)
```

## GPU and Optimization

```python
# GPU inference
session = ort.InferenceSession("model.onnx", providers=["CUDAExecutionProvider"])

# Enable optimizations
options = ort.SessionOptions()
options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
session = ort.InferenceSession("model.onnx", sess_options=options)
```

## Quantization

```python
from onnxruntime.quantization import quantize_dynamic, QuantType

quantize_dynamic("model.onnx", "model_quantized.onnx", weight_type=QuantType.QInt8)
# ~4x smaller, minimal accuracy loss
```

## References
- [ONNX Runtime docs](https://onnxruntime.ai/docs/)
- [ONNX model zoo](https://github.com/onnx/models)