open-edge-platform Geti Runtime: Running Live Inference
Use this skill for operational control of live inference in Geti. This skill is for runtime execution and monitoring, not code changes.
When to Use
- User asks how to start or stop live inference for a configured pipeline.
- User needs runbook-style checks for runtime health and correctness.
- User reports stalls, dropped outputs, or latency spikes during live execution.
- User needs incident-style recovery steps with minimal disruption.
Scope
- In scope: runtime start and stop flow, status monitoring, health verification, rollback and recovery.
- Out of scope: backend implementation changes and model retraining.
Procedure
- Pre-flight checks.
- Confirm pipeline configuration is complete (source, model, sink).
- Confirm model is loaded and selected.
- Confirm endpoints for source and sink are reachable.
- Start live inference.
- Enable runtime execution through the application pipeline workflow.
- Capture execution identifiers and current status.
- Monitor execution.
- Check the pipeline status (Idle vs Running).
- Verify frames are rendered with predictions, inference output is generated as per sink configuration.
- Record key runtime metrics if available (throughput & latency).
- Validate output quality.
- Spot-check predictions against known scenes or samples.
- Confirm class distribution and confidence values are reasonable.
- Handle degradation.
- If source degrades: reconnect or switch to a known-good source.
- If model output degrades: verify loaded model version and thresholds.
- If sink fails: apply fallback sink.