# Review Topic Local Vs Remote Execution

> Load when a PR touches how a step/model runs in-process vs. over the network: Workflow-block execution-mode dispatch (self._step_execution_mode, StepExecutionMode.LOCAL/REMOTE, run_locally/run_remotely), a block get_restrictions()/get_air_gapped_availability(), backend/runtime routing env (USE_INFERENCE_MODELS, DISABLED_INFERENCE_MODELS_BACKENDS, WORKFLOWS_REMOTE_API_TARGET, inference_models_adapters.py), CUDA/Jetson/ONNX gating, or a new block with network/LAN/same-process assumptions.

- Skill: `roboflow/review-topic-local-vs-remote-execution` (Agent Skill)
- Install (CLI): `npx skillmds@latest add roboflow/review-topic-local-vs-remote-execution`
- Raw SKILL.md: https://api.skillmd.com/api/skills/roboflow/review-topic-local-vs-remote-execution/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: roboflow (https://skillmd.com/u/roboflow)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/roboflow/review-topic-local-vs-remote-execution

---


# Review topic: Local vs remote / hosted execution compatibility

## When this applies
Load when a diff does any of:
- Adds/edits a `run()` dispatch on `self._step_execution_mode` (LOCAL vs REMOTE) or a `run_locally` / `run_remotely` pair.
- Adds/edits a block's `get_restrictions()` or `get_air_gapped_availability()` where the block reaches the network or needs same-process / LAN access (PTZ/ONVIF camera, local filesystem, GPU-resident handles).
- Touches runtime routing env: `USE_INFERENCE_MODELS`, `DISABLED_INFERENCE_MODELS_BACKENDS`, `WORKFLOWS_STEP_EXECUTION_MODE`, `WORKFLOWS_REMOTE_API_TARGET`, `LOCAL_INFERENCE_API_URL`, `_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT`, or backend selection in `inference/core/models/inference_models_adapters.py` / `inference/models/utils.py`.
- Gates behavior on hardware/deps: CUDA version, Jetson (`jetson`, `l4t`, JP5/JP6), ONNX execution providers, TRT — where one runtime path is fixed/tested but siblings are not.
- Changes hosted/serverless/dedicated cold-start / processing-time headers or WebRTC/modal worker plumbing.

This is a behavior/content trigger, not a single directory — the paths above are hints.

**Cross-refs (owned elsewhere, not reviewed here):**
- Serverless auth/usage fail-open behavior → `review-topic-auth-and-tenant-security`.
- Stateful-block restriction contract (in-process trackers/counters/cooldown/append-log needing a SOFT restriction) → `review-topic-workflow-state-management`.

## Review checklist

- **BLOCK** — `run()` dispatches on `StepExecutionMode` with an explicit `else: raise ValueError` for unknown modes (missing else swallows new modes silently).
- **BLOCK** — a runtime-routing change (`USE_INFERENCE_MODELS`, `DISABLED_INFERENCE_MODELS_BACKENDS`, platform default) leaves BOTH the on and off value working and tested; the legacy ONNX registry entries are still present. Breaking the DEFAULT runtime with no working fallback is a production regression.
- **BLOCK** — a block that raises / hard-fails on a runtime the platform will still offer carries a matching `RuntimeRestriction` (right `severity`, `applies_to_step_execution_modes`, `applies_to_runtimes`); metadata that lies makes the UI offer a path that fails at runtime.
- **FLAG** — `run_locally` and `run_remotely` have matching signatures and forward every manifest param; a param wired into only one path is a divergence users hit only on the other runtime.
- **FLAG** — a new network/LAN/same-process block overrides `get_air_gapped_availability()` if it needs internet, and adds a HARD restriction excluding hosted/dedicated (LAN) or REMOTE step execution (same-process).
- **FLAG** — a platform-specific default (`_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT`, Windows/Jetson forks) keeps the non-default OS working; the diff must not assume one platform.
- **FLAG** — CUDA/Jetson/ONNX-provider introspection changes are correct on every runtime, not just the author's box (one-runtime introspection bugs mis-route model loading).
- **NIT** — restriction present but mis-scoped or wrong severity (HARD where SOFT fits, missing `applies_to_runtimes`).

### Not blocking
- Do NOT demand a hosted-platform test for a change with no observable hosted/serverless behavior difference (pure local refactor, docstrings, typing). Hosted coverage under `tests/inference/hosted_platform_tests/` is required only when the diff changes what a request returns on lambda / serverless / dedicated.
- Do NOT demand a restriction for a block that is genuinely runtime-agnostic (stateless, no network, no LAN, no GPU-handle assumption).
- Do NOT demand a flip-the-flag-both-ways test for env vars that only tune values (URLs, timeouts) without forking code paths.

## Standards

1. **Execution-mode dispatch is exhaustive.** `run()` branches on `self._step_execution_mode` and terminates in `else: raise ValueError(f"Unknown step execution mode: ...")`. Canonical shape: `inference/core/workflows/core_steps/models/roboflow/object_detection/v1.py` — `run()` branches `StepExecutionMode.LOCAL` / `StepExecutionMode.REMOTE`, `run_locally` via `self._model_manager`, `run_remotely` via `InferenceHTTPClient` honoring `WORKFLOWS_REMOTE_API_TARGET` / `LOCAL_INFERENCE_API_URL`. (`StepExecutionMode` is defined in `inference/core/workflows/prototypes/block.py` and re-exported from `core_steps/common/entities.py`.)

2. **Local and remote paths stay semantically equivalent.** `run_locally` and `run_remotely` accept the same params and return the same shape; every manifest field is forwarded on both paths (one-path param wiring is a silent divergence).

3. **Metadata must match behavior.** A block's `get_restrictions()` / `get_air_gapped_availability()` declarations are consumed by the describe endpoint and the platform UI to decide where a block may run. The declaration and the actual `run()` guard must agree — a code path that raises for a runtime needs a `RuntimeRestriction` for it, and vice-versa. LAN shape: `onvif_movement/v1.py::NO_LAN_FROM_HOSTED_RESTRICTION` (HARD + `Runtime.HOSTED_SERVERLESS`/`Runtime.DEDICATED_DEPLOYMENT` because hosted can't reach a customer LAN). Do NOT gate a non-model block on `StepExecutionMode` - that enum only says where model steps run; a block with no remote path always executes in-process, and a REMOTE-scoped restriction is silent under the default LOCAL mode on hosted runtimes.

4. **Runtime routing has no orphaned path.** A new backend, env flag, or hardware gate leaves BOTH the on and off value working and tested; the legacy path (ONNX registry entries) stays intact. Backend routing lives in `inference/core/models/inference_models_adapters.py` + `inference/core/env.py` (`USE_INFERENCE_MODELS`, `_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT`, `DISABLED_INFERENCE_MODELS_BACKENDS`). Rollout history: #2096 added per-deployment `DISABLED_INFERENCE_MODELS_BACKENDS`; #2128 added missing registry entries so RFDETR-NAS works under inference-models; #2136 reverted making inference-models the default backend (the default flip is contentious — review carefully).

5. **Per-platform introspection is correct on every runtime.** CUDA-version / Jetson / ONNX-provider detection that gates model loading must hold across platforms, not just the author's (CUDA acquisition fixed in #2515; Jetson environment detection in runtime introspection + dockerfiles fixed in #1796).

6. **Hosted-only code paths get hosted coverage.** Behavior that differs on lambda / serverless / dedicated is exercised in `tests/inference/hosted_platform_tests/` (parametrized over `PlatformEnvironment` via `conftest.py`; see `test_workflows.py`, `test_roboflow_models.py`). Hosted-only plumbing regresses silently under local-only tests (remote cold-start header aggregation reverted in #2222; `ROBOFLOW_ENVIRONMENT` propagation to the modal webrtc worker fixed in #2179; dedicated `hosted=True`/`DEDICATED_DEPLOYMENT_ID` usage attribution fixed in #817).

## Key files & Reference PRs

Files:
- `inference/core/workflows/prototypes/block.py` — the contract: `StepExecutionMode`, `Runtime`, `Severity`, `RuntimeRestriction`, presets (`STATEFUL_VIDEO_HTTP_SOFT_RESTRICTION`, `COOLDOWN_HTTP_SOFT_RESTRICTION`), and `get_restrictions` / `get_air_gapped_availability` / `get_execution_engine_compatibility` hooks. Read first.
- `inference/core/workflows/core_steps/models/roboflow/object_detection/v1.py` — canonical LOCAL/REMOTE dispatch.
- `inference/core/workflows/core_steps/sinks/onvif_movement/v1.py` — LAN block done right (one HARD runtime-keyed restriction, shared with `v1_tensor.py`; no `step_execution_mode` gate).
- `inference/core/workflows/core_steps/cache/common.py` — in-process cache caveat done right (SOFT, runtime-keyed, not narrowed by step mode or input mode).
- `inference/core/models/inference_models_adapters.py` + `inference/core/env.py` — backend routing with per-platform default and per-deployment disable.
- `tests/inference/hosted_platform_tests/conftest.py` (+ `test_workflows.py`, `test_roboflow_models.py`) — hosted/serverless/localhost parametrization.

Reference PRs: #2096, #2128, #2136 (backend rollout / default flip), #2515, #1796 (per-platform introspection), #2222, #2179, #817 (hosted-only plumbing).