Use Iris
Read only the material needed for the request:
- Normal jobs, tasks, scheduling, auth, or CoreWeave:
lib/iris/OPS.md. - Federation:
lib/iris/docs/federation.md. - Continuous job monitoring: references/monitor-job.md.
- Controller deploy or rollback: references/controller-rollout.md.
- Interactive GPU or TPU: references/dev-accelerators.md.
- Stuck terminating CoreWeave pod: references/stuck-pod.md.
- Temporary task outputs:
lib/iris/docs/task-outputs.md. - Logs or measurements: use
query-finelog.
Resolve cluster facts from lib/iris/config/<cluster>.yaml; do not copy live coordinates from memory.
Common reads
uv run iris --cluster=<cluster> job describe <job>
uv run iris --cluster=<cluster> task describe <task>
uv run iris --cluster=<cluster> task events <task>
uv run iris --cluster=<cluster> rpc controller list-backends
For a pending federated root, inspect all three parent-side views:
uv run iris --cluster=<parent> job list --prefix <root-job>
uv run iris --cluster=<parent> rpc controller list-peers
uv run iris --cluster=<parent> query \
"SELECT job_id, peer_id, handoff_state FROM federated_jobs WHERE job_id='<root-job>'"
Only root jobs federate; their whole tree stays on the peer. Parent job describe is the liveness source, while forwarded logs may lag. CoreWeave tasks normally read regional S3 and GCP tasks read GCS.
Temporary outputs
Write bounded diagnostics to $IRIS_OUTPUT_DIR. Iris preserves that directory as one outputs.tar.zst archive per attempt without changing the command outcome when capture fails. Find the archive URI and its uploaded, empty, failed, or unavailable state with:
uv run iris --cluster=<cluster> attempt describe <task>:<attempt>
Use direct object-storage writes for large or durable outputs. See lib/iris/docs/task-outputs.md for retention, limits, and data-access boundaries.
Boundaries
- Start read-only and name the evidence that distinguishes each cause.
- Never run
iris cluster restartwithout explicit approval for the named cluster; it kills all workers and jobs. - Treat a controller restart as a deployment and require an explicitly named target.
- Cancel, complete, fail, preempt, resubmit, or change Kubernetes state only when the request or selected reference authorizes that exact action.
- Avoid
kubectl describe podon task pods because it can print environment values.