Ito Inference

Itô Inference

ito-inference is the sole canonical ECC skill for inference serving on Itô compute. Requests naming ito-serve route here; do not create or install a second ito-serve skill. ECC never SSHes to nodes, downloads weights, launches an engine, or exposes an endpoint; it never books, reserves, or spends.

Current production boundary

Managed serving is unavailable today. The ECC bridge exposes only login, auth, find, status, and explicitly gated evals. It has no serve verb. The canonical runtime documents inference only as an unsupported compatibility probe; ECC does not invoke or depend on it. The MCP surface exposes only auth, find, and status. The locally enforceable guarantee is that ECC rejects serve before resolving or spawning the credential-bearing canonical client.

Therefore stop before authentication or any command invocation. Report the missing capability and return to the originating agent. Never substitute a local runner, SSH helper, browser workflow, purchase endpoint, or any untracked local ito-serve draft.

Required entitlement

When serving is implemented, its first gate is a server-verified completed booking. Harness memory, an RFQ, a quote, node IPs, or SSH access are not proof of entitlement. The backend must return fresh serving eligibility bound to the authenticated account, booking, GPU topology, region, fabric, term, and model policy. Expired, revoked, mismatched, incomplete, or already-released bookings fail closed before confirmation.

Future CLI and API contract

The intended command name is serve; inference may remain only as an explicitly deprecated compatibility alias after the production contract lands. The future handoff must be equivalent to:

ecc ito serve \
  --booking <server-verified-booking-id> \
  --manifest <absolute-reviewed-json-file> \
  --confirmation-ref <opaque-non-authorizing-reference> \
  --idempotency-key <stable-retry-key> \
  --json

The reviewed manifest must identify the model revision, engine and version, quantization, tensor/pipeline topology, endpoint exposure policy, artifact checksums, storage ceiling, runtime limits, optional TTFT/TPOT objectives, and maximum incremental cost. No raw API key, SSH key, node password, or bearer token belongs in arguments, manifests, logs, MCP results, or chat.

The client must canonicalize the manifest path, reject symlinks, open a regular file without following links, require appropriate ownership and restrictive permissions, enforce a bounded size, and hash bytes from the opened descriptor. That digest must exactly equal the digest bound into confirmation before any workload mutation. A path swap, digest mismatch, oversized file, or mutable unsafe file fails closed.

The canonical API—not ECC—must own workload creation and return structured JSON with ok, live_api_contacted, notice, and either data or error. Serving data must include stable booking, workload, manifest, and idempotency IDs plus a state enum; it must not claim an endpoint is live until health and model checks pass. Errors must include a stable code and safe message without secrets.

Confirmation and execution gates

Before workload creation, require all of the following:

1. Fresh entitlement and serving eligibility from the canonical backend.
2. A reviewable immutable manifest and deterministic digest.
3. A separate single-use confirmation bound to account, action, manifest, and cost, with a short expiry and replay protection. CLI arguments carry only an opaque, non-authorizing confirmation reference; the server resolves and consumes the bearer capability out of band.
4. A caller-supplied idempotency key reserved atomically with the workload.
5. Server-side fabric, capacity, model-policy, storage, and cost validation.

Authentication is identity, not workload authority. A login, API key, quote, or completed booking never substitutes for the serving confirmation. Inspection and plan generation must not create a workload. Cancel and cleanup are separate mutations with their own scoped confirmation and idempotency boundaries.

Lifecycle and recovery

The production surface is incomplete until the same canonical client exposes tenant-scoped status, logs, metrics, cancel, and cleanup operations. Every operation needs bounded connect and overall timeouts, revocation-aware errors, and structured output. After an ambiguous transport failure, query status by the idempotency key before retrying; never create a second workload merely because the first response was lost. A revoked credential stops polling and returns control to the originating agent without starting login automatically.

Only report ready after endpoint health, model identity, and canary inference all pass. Report intermediate and terminal failure states honestly. Cleanup must be observable and must not release or modify the underlying booking unless that separate economic action was explicitly authorized.

Proposed backend stages

These stages describe the future backend, not code that exists in ECC:

1. Verify entitlement, topology, fabric, and cost gates.
2. Fetch checksum-pinned weights into backend-managed storage.
3. Emit and validate a reviewable topology/engine plan.
4. Launch through the provider control plane, never direct root SSH from ECC.
5. Warm up, test health and model identity, run an SLO canary, then register the endpoint and redacted configuration.

Until every gate and lifecycle operation above exists in the canonical runtime, this skill remains a fail-closed availability check and documentation handoff.