Image generation through gateways
A gateway standardizes account access, not model behavior. Keep gateway, model publisher, model/version, execution path, schema, price, safety behavior, and downstream terms as separate facts in every plan and ledger row.
This skill covers three current multi-model serverless gateways:
- fal.ai Model APIs: direct or persistent queue calls to model endpoint IDs, with queue status, cancellation, and signed webhooks.
- Replicate predictions: prediction resources around official, community, marketplace, or version-pinned models.
- Together AI serverless images: a synchronous OpenAI-compatible image endpoint spanning models with different parameter subsets.
These gateways were selected because their first-party contracts expose three materially different operating patterns. Do not add another gateway merely because it offers the same model name. Research its control plane, billing unit, data path, lifecycle, schema, and terms first.
Volatile facts here were checked 2026-07-10. Use four evidence labels in user-facing work:
- FACT: directly documented by the cited gateway, model publisher, or governing terms.
- PROVIDER CLAIM: a quality, commercial-use, training, security, equivalence, or infrastructure representation that was not independently audited here.
- HEURISTIC: a conservative production practice in this skill, not a gateway limit.
- UNKNOWN: public first-party evidence was insufficient; resolve through the account, contract, DPA, model publisher, or support.
Build the routing record before code
Capture a signed-off record with:
- production intent and image acceptance tests;
- gateway and account/project;
- exact endpoint/model ID, publisher, official/community/partner status, and version policy;
- schema URL or API-discovered schema plus retrieval time and hash;
- complete input fields, reference provenance, output count/format/size, safety controls, and seed;
- execution path: sync, queue/poll, or webhook;
- billing unit, unit price snapshot, estimated maximum, credits/discounts, and approver;
- input/output rights, model terms, gateway terms, downstream processor/data path, retention, training setting, and region;
- artifact destination, byte/pixel limits, provenance fields, and deletion schedule;
- retry, reconciliation, cancellation, and failover policy.
If any consequential entry is missing, prepare a dry run and stop before submission.
Select for the workload, not the logo
| Requirement | fal.ai | Replicate | Together AI |
|---|---|---|---|
| Long-running serverless control | Queue returns request/status/response/cancel URLs; webhooks available | Async prediction is default; poll, cancel, or webhook | Current /v1/images/generations is synchronous; no image-job resource documented |
| Version behavior | Endpoint ID does not by itself prove an immutable model revision | Community predictions can use an immutable version ID; official model endpoint follows maintained latest | Model string may be deprecated or redirected; dedicated endpoint is the documented escape hatch for an original version |
| Schema discovery | Every model page exposes its own schema; common fields are not platform fields | models.get exposes latest_version.openapi_schema; version pages preserve version-specific schema |
/v1/models exposes model metadata, but image parameter support still requires image docs and model-specific pages |
| Output path | Public fal.media CDN URLs unless a model returns inline data |
Current clients expose FileOutput; URLs use replicate.delivery and expire after one hour |
Request response_format: base64 to avoid a second URL fetch |
| Data controls | X-Fal-Store-IO: 0; separate CDN lifecycle control |
API prediction inputs, outputs, logs, and files removed after one hour by default | Account ZDR/privacy setting; public documents conflict on the default, so verify it |
| Hidden routing risk | Queue retries and supported-model fallbacks are enabled by default | Official model can advance to a new version | Deprecation policy permits redirects for selected model IDs |
No cell means "best." Benchmark the actual brief using fixed acceptance tests. The same seed across gateways or model builds is not comparable reproducibility.
Freeze each model contract
Do not implement a universal payload containing prompt, width, steps, and seed and send it everywhere. For every approved adapter:
- retrieve the current model-specific schema from the first-party page/API;
- permit only fields in that schema, with local type/range checks;
- save a canonical schema hash and model/page URL in the release record;
- test a dry request fixture and mocked success/failure responses;
- require review when the schema hash, endpoint ID, returned model/version, price unit, publisher terms, or safety defaults change.
fal.ai facts that change design
- REST authentication is
Authorization: Key <FAL_KEY>. - The production queue for endpoint
fal-ai/flux/schnellisPOST https://queue.fal.run/fal-ai/flux/schnell; queue states areIN_QUEUE,IN_PROGRESS, andCOMPLETED, after which the response URL is retrieved. - Queue requests can retry up to ten times on selected server/connection failures. For supported models, fal may route to an "equivalent" fallback endpoint after retries. This is a PROVIDER CLAIM of equivalence, not proof of identical model, weights, schema, license, safety, or output.
- Use
X-Fal-No-Retry: 1andx-app-fal-disable-fallback: 1when one approved submission must mean one approved endpoint execution. A client timeout does not stop server processing. X-Fal-Store-IO: 0prevents platform JSON input/output history storage, but not CDN media. Default JSON payload retention is documented as 30 days.X-Fal-Object-Lifecycle-Preferencecontrols generated-media expiration; account default may be forever if not configured.- fal media URLs are public to anyone holding the URL. Files uploaded as inputs to fal CDN are not removed when request payloads are deleted.
- Model pricing is per endpoint and may be per image, megapixel, or compute second. Query
GET https://api.fal.ai/v1/models/pricing?endpoint_id=...and use account-specific results. - fal webhooks use ED25519 with public keys from
https://rest.fal.ai/.well-known/jwks.json, fourX-Fal-Webhook-*headers, SHA-256 of the raw body, and a documented +/-300-second timestamp check.
Replicate facts that change design
- REST authentication is
Authorization: Bearer <REPLICATE_API_TOKEN>. - Generic pinned prediction:
POST https://api.replicate.com/v1/predictionswithversionandinput. Official model:POST /v1/models/{owner}/{name}/predictionswithout a version; Replicate keeps it current. - Prediction states are
starting,processing,succeeded,failed, andcanceled. Async is default.Prefer: wait=nonly holds the HTTP call for 1-60 seconds and can still return an incomplete prediction.Cancel-Aftersets a server deadline from 5 seconds to 24 hours. - API prediction input, output, files, and logs are deleted after one hour by default. Web-created prediction data is kept indefinitely until deletion.
- Official models have a stable API and output-based price, but the backing version is maintained latest. Community versions can be pinned, but have creator-controlled support/schema and commonly runtime/hardware billing.
- The current output-file guide says SDK
FileOutputshould replace URL/auth handling and thatreplicate.deliveryURLs need no Authorization logic. The general HTTP reference still says file URLs need Authorization. Treat this as a documentation conflict: prefer current SDKFileOutput, or fetch only documentedreplicate.deliveryURLs without forwarding the API token; do not "fix" a 401 by attaching credentials to an arbitrary host. - Replicate webhooks are HMAC-SHA256 over
{webhook-id}.{webhook-timestamp}.{raw-body}, using the base64 portion of the per-user/orgwhsec_secret. Deliveries can duplicate and arrive out of order; terminal deliveries retry on backoff until roughly one minute after completion. - Current default limits are documented as 600 prediction creates/minute and 3000 other calls/minute, with stricter low-credit conditions. Live 429 recovery text controls.
Together AI facts that change design
- Current base is
https://api.together.ai/v1; use Bearer authentication. Older first-party model pages still showapi.together.xyz, so use the current API/compatibility docs and SDK default. POST /v1/images/generationsreturns the image response synchronously. A transport timeout is an ambiguous create; no public idempotency key or image-job lookup was found.- The endpoint has common fields, but availability differs by model. The image overview says FLUX Schnell uses
aspect_ratio, while the current FLUX Schnell model page demonstrateswidth/heightand the endpoint reference exposes those fields. Treat this as a first-party documentation conflict, freeze a tested schema/SDK version, and do not infer support.image_urlandreference_imagesalso apply to different model sets. response_formatisurlorbase64;nis documented as 1-4. Keepdisable_safety_checkerfalse. Some models do not run the gateway safety checker, so application policy is still required.- Serverless image pricing is model-specific, generally per image or megapixel, and steps above a listed default may add cost. Dynamic model-specific rate limits are returned in response headers.
- The deprecation policy documents both no-redirect removals and active redirects. Compare the returned
modelwith the requested model and quarantine a mismatch. A model string is not always an immutable revision. - Together's Privacy and Security page says it does not store inputs/outputs by default, while its current Privacy Policy and Terms describe enabling ZDR by selecting "No" for storage/training. Treat the default as UNKNOWN; verify the project setting and executed contract. ZDR is prospective, not retroactive.
Paid-call barrier
Schema reads, pricing reads, plans, and local/mocked validation do not authorize generation. Free endpoints and prepaid credits still consume a limited resource.
Before every live generation require a canonical UUIDv4 attempt, a same-day price/schema/terms snapshot, positive reviewed unit price and estimate, and a finite positive ceiling. Use the bundled Python 3.11+ stdlib helper scripts/validate_plan.py to validate the offline approval plan before any submit-capable adapter runs:
python scripts/validate_plan.py approval-plan.json --max-age-days 1
python scripts/validate_plan.py approval-plan.json --max-age-days 1 --expected-approval-sha256 <64-hex-digest>
The helper never imports provider SDKs, reads credentials, makes network calls, generates media, authorizes a paid request, or submits a job. It checks gateway/model/version policy, schema URL and SHA-256, canonical payload shape, rejects unknown keys at every defined approval-plan object level, scans the full plan for likely secret-bearing keys or values, rejects non-standard JSON constants such as NaN and Infinity, output count and policy, credential-free HTTPS schema/price URLs, price source/freshness/currency/unit/unit price/billable units, estimate == unit_price * billable_units using decimal arithmetic, ceiling >= estimate, rights/moderation/governance digests, and a canonical UUIDv4 attempt. The documented safe fal pricing query parameter endpoint_id is allowed in price.source; secret-bearing query parameters and secret-looking query values are not. It emits stable redacted JSON and computes approval_sha256 over the documented approval envelope in that output, so every accepted semantic field affects the digest. Exit code 0 means the plan is valid, 2 means validation or expected-digest mismatch failed, and 3 means parse or operational failure.
Hash one canonical authorization record containing the attempt, approval reference, exact endpoint/model/payload/count, output destination and policy, schema hash, price source/unit/floor/estimate/ceiling, and governance dispositions. GATEWAY_APPROVAL_SHA256 must exactly equal that record; a free-form ticket string is not the gate.
Create the attempt ledger with O_CREAT|O_EXCL before POST. The attempt UUID and approval digest are single-use. Persist the provider ID immediately, before polling, output validation, or asset download. A later process may use GATEWAY_RESUME=1 only for the same durable fal/Replicate identity; a Together timeout or an attempt without an ID must be reconciled rather than resubmitted.
Never auto-generate exploratory variants. A retry or cross-gateway failover is a new approved create unless the provider proves it is the same job.
Example price observations, not defaults (2026-07-10): fal documents fal-ai/flux/schnell at $0.003/MP; Replicate's official black-forest-labs/flux-schnell page shows $3/1000 output images; Together lists black-forest-labs/FLUX.1-schnell at $0.0027/MP with four default steps. Account discounts, rounding, requested size/count/steps, model revisions, and taxes can change the estimate. Query again.
Complete adapter example
The following Python 3.11+ example is deliberately three adapters, not one translated payload. It uses text-to-image only so reference-upload policy cannot be mistaken for a universal contract. It defaults to an offline plan, binds exact approval and positive spend floors, preflights Replicate model/version OpenAPI, writes an exclusive pre-create ledger, disables fal retries/fallback, resumes only known async IDs, requests Together base64, refuses redirects, validates image bytes with Pillow, and writes a sanitized release/billing/provenance manifest.
Install Pillow for decode checks: python -m pip install Pillow. Copy as gateway_image.py.
from __future__ import annotations
import base64
import binascii
import hashlib
import ipaddress
import json
import math
import os
import platform
import random
import socket
import tempfile
import time
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
from decimal import Decimal, InvalidOperation
from email.utils import parsedate_to_datetime
from io import BytesIO
from pathlib import Path
from typing import Any, Callable
from urllib.error import HTTPError, URLError
from urllib.parse import urlsplit
from urllib.request import HTTPRedirectHandler, Request, build_opener
JSON_CAP = 48 * 1024 * 1024
IMAGE_CAP = 48 * 1024 * 1024
PIXEL_CAP = 36_000_000
POLL_DEADLINE_S = 360
CREATE_DEADLINE_S = 90
ASSET_DEADLINE_S = 120
READ_SLICE_S = 5
SCHEMA_CHECKED = "2026-07-10"
PRICE_CHECKED = "2026-07-10"
OUTPUT_POLICY_VERSION = "image-v1"
class StopRedirects(HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
raise RuntimeError(f"redirect refused: {code}")
HTTP = build_opener(StopRedirects)
class GatewayHTTPError(RuntimeError):
def __init__(self, code: int, safe_message: str):
super().__init__(safe_message)
self.code = code
class GatewayTerminalError(RuntimeError):
pass
class AmbiguousCreate(RuntimeError):
pass
@dataclass(frozen=True)
class Plan:
gateway: str
model: str
url: str
payload: dict[str, Any]
lifecycle: str
version_policy: str
schema_url: str
price_source: str
price_unit: str
known_unit_price: Decimal
billable_units: Decimal
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat()
def canonical(value: Any) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode()
def sha256_value(value: Any) -> str:
return hashlib.sha256(canonical(value)).hexdigest()
def require_sha256(name: str, value: str) -> str:
value = value.strip().lower()
if len(value) != 64 or any(char not in "0123456789abcdef" for char in value):
raise ValueError(f"{name} must be a 64-character lowercase hex SHA-256")
return value
def parse_positive_money(name: str, value: str, required: bool) -> Decimal | None:
value = value.strip()
if not value:
if required:
raise ValueError(f"{name} is required")
return None
try:
amount = Decimal(value)
except InvalidOperation as exc:
raise ValueError(f"{name} must be a finite positive decimal") from exc
if not amount.is_finite() or amount <= 0:
raise ValueError(f"{name} must be a finite positive decimal")
return amount
def valid_attempt_id(value: str) -> str:
try:
parsed = uuid.UUID(value)
except ValueError as exc:
raise ValueError("GATEWAY_ATTEMPT_ID must be a UUIDv4") from exc
normalized = str(parsed)
if parsed.version != 4 or value.lower() != normalized:
raise ValueError("GATEWAY_ATTEMPT_ID must be a canonical UUIDv4")
return normalized
def valid_provider_id(value: Any, provider: str) -> str:
if not isinstance(value, str) or not value or len(value) > 256:
raise AmbiguousCreate(f"{provider} create returned no usable identity")
if any(char not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-" for char in value):
raise AmbiguousCreate(f"{provider} create returned an unsafe identity")
return value
def public_https(url: str, allowed: tuple[str, ...]) -> str:
parsed = urlsplit(url)
host = (parsed.hostname or "").lower()
if parsed.scheme != "https" or not host or parsed.username or parsed.password or parsed.fragment:
raise ValueError("asset must be credential-free fragment-free HTTPS")
if parsed.port not in (None, 443):
raise ValueError("asset port is not allowed")
if not any(host == suffix or host.endswith("." + suffix) for suffix in allowed):
raise ValueError("asset host is not provider-approved")
for answer in socket.getaddrinfo(host, 443, type=socket.SOCK_STREAM):
if not ipaddress.ip_address(answer[4][0]).is_global:
raise ValueError("asset resolved to a non-public address")
return url
def api_url(url: str, host: str, prefix: str) -> str:
parsed = urlsplit(url)
expected_path = parsed.path == prefix or parsed.path.startswith(prefix.rstrip("/") + "/")
if (
parsed.scheme != "https"
or parsed.hostname != host
or parsed.port not in (None, 443)
or parsed.username
or parsed.password
or parsed.fragment
or not expected_path
):
raise ValueError("unexpected API URL")
return url
def retry_after_seconds(headers: Any, now: datetime | None = None) -> float | None:
value = headers.get("Retry-After") if headers is not None else None
if not value:
return None
try:
seconds = float(value)
return max(0.0, min(seconds, 60.0)) if math.isfinite(seconds) else None
except ValueError:
try:
target = parsedate_to_datetime(value)
if target.tzinfo is None:
target = target.replace(tzinfo=timezone.utc)
current = now or datetime.now(timezone.utc)
return max(0.0, min((target - current).total_seconds(), 60.0))
except (TypeError, ValueError, OverflowError):
return None
def bounded_read(response: Any, cap: int, deadline: float, expected_type: str | None) -> bytes:
if time.monotonic() >= deadline:
raise TimeoutError("response deadline reached")
length = response.headers.get("Content-Length")
if length:
try:
declared = int(length)
except ValueError as exc:
raise RuntimeError("invalid Content-Length") from exc
if declared < 0 or declared > cap:
raise RuntimeError("response exceeds client cap")
if expected_type and response.headers.get_content_type() != expected_type:
raise RuntimeError("unexpected response content type")
chunks: list[bytes] = []
size = 0
reader = getattr(response, "read1", response.read)
while True:
if time.monotonic() >= deadline:
raise TimeoutError("response deadline reached")
chunk = reader(min(64 * 1024, cap + 1 - size))
if not chunk:
break
size += len(chunk)
if size > cap:
raise RuntimeError("response exceeds client cap")
chunks.append(chunk)
return b"".join(chunks)
def safe_error_hash(exc: HTTPError, deadline: float) -> str:
try:
raw = bounded_read(exc, 16_384, deadline, None)
except Exception:
raw = b""
finally:
try:
exc.close()
except OSError:
pass
return hashlib.sha256(raw).hexdigest()
def post_json(
url: str,
expected_host: str,
path_prefix: str,
auth_header: str,
payload: dict[str, Any],
extra_headers: dict[str, str] | None = None,
) -> dict[str, Any]:
api_url(url, expected_host, path_prefix)
body = json.dumps(payload, separators=(",", ":")).encode()
headers = {
"Accept": "application/json",
"Authorization": auth_header,
"Content-Type": "application/json",
}
headers.update(extra_headers or {})
deadline = time.monotonic() + CREATE_DEADLINE_S
try:
timeout = max(0.1, min(READ_SLICE_S, deadline - time.monotonic()))
with HTTP.open(Request(url, data=body, headers=headers, method="POST"), timeout=timeout) as response:
raw = bounded_read(response, JSON_CAP, deadline, "application/json")
value = json.loads(raw)
if not isinstance(value, dict):
raise RuntimeError("gateway create response is not an object")
return value
except HTTPError as exc:
error_hash = safe_error_hash(exc, deadline)
wait = retry_after_seconds(exc.headers)
suffix = f"; retry_after={wait:g}s" if wait is not None else ""
if exc.code >= 500:
raise AmbiguousCreate(
f"create returned HTTP {exc.code}; body_sha256={error_hash}; reconcile before resubmission"
) from exc
raise GatewayHTTPError(
exc.code, f"gateway rejected create with HTTP {exc.code}; body_sha256={error_hash}{suffix}"
) from None
except GatewayHTTPError:
raise
except AmbiguousCreate:
raise
except Exception as exc:
raise AmbiguousCreate(
"create transport or 2xx response failed before a durable identity; do not resubmit"
) from exc
def sleep_with_deadline(seconds: float, deadline: float) -> None:
remaining = deadline - time.monotonic()
if remaining <= 0:
raise TimeoutError("gateway deadline reached")
time.sleep(min(max(0.0, seconds), remaining))
def get_json(
url: str,
expected_host: str,
path_prefix: str,
auth_header: str,
deadline: float,
event: Callable[[dict[str, Any]], None] | None = None,
) -> dict[str, Any]:
api_url(url, expected_host, path_prefix)
delay = 1.0
while True:
if time.monotonic() >= deadline:
raise TimeoutError("read-only gateway deadline reached")
headers = {"Accept": "application/json", "Authorization": auth_header}
try:
timeout = max(0.1, min(READ_SLICE_S, deadline - time.monotonic()))
with HTTP.open(Request(url, headers=headers, method="GET"), timeout=timeout) as response:
raw = bounded_read(response, JSON_CAP, deadline, "application/json")
value = json.loads(raw)
if not isinstance(value, dict):
raise RuntimeError("gateway response is not an object")
return value
except HTTPError as exc:
retryable = exc.code == 429 or exc.code >= 500
error_hash = safe_error_hash(exc, deadline)
if not retryable:
raise GatewayHTTPError(
exc.code, f"gateway read failed with HTTP {exc.code}; body_sha256={error_hash}"
) from None
wait = retry_after_seconds(exc.headers)
pause = wait if wait is not None else delay + random.uniform(0, delay * 0.15)
if event:
event({"kind": "read_retry", "http_status": exc.code, "wait_seconds": pause})
except (TimeoutError, URLError, RuntimeError, json.JSONDecodeError) as exc:
if time.monotonic() >= deadline:
raise TimeoutError("read-only gateway deadline reached") from exc
pause = delay + random.uniform(0, delay * 0.15)
if event:
event({"kind": "read_retry", "error": type(exc).__name__, "wait_seconds": pause})
sleep_with_deadline(pause, deadline)
delay = min(delay * 1.7, 12.0)
def check_image(blob: bytes) -> tuple[str, str, int, int]:
if not blob or len(blob) > IMAGE_CAP:
raise ValueError("image is empty or exceeds client cap")
if blob.startswith(b"\x89PNG\r\n\x1a\n"):
kind, suffix = "image/png", ".png"
elif blob.startswith(b"\xff\xd8\xff"):
kind, suffix = "image/jpeg", ".jpg"
elif blob.startswith(b"RIFF") and blob[8:12] == b"WEBP":
kind, suffix = "image/webp", ".webp"
else:
raise ValueError("output magic is not PNG, JPEG, or WebP")
try:
from PIL import Image, UnidentifiedImageError
Image.MAX_IMAGE_PIXELS = PIXEL_CAP
with Image.open(BytesIO(blob)) as image:
width, height = image.size
if width * height > PIXEL_CAP or getattr(image, "n_frames", 1) != 1:
raise ValueError("output exceeds pixel/frame policy")
image.verify()
with Image.open(BytesIO(blob)) as image:
image.load()
except ImportError as exc:
raise RuntimeError("Pillow is required for artifact validation") from exc
except (UnidentifiedImageError, OSError) as exc:
raise ValueError("output failed full image decode") from exc
return kind, suffix, width, height
def fetch_asset(url: str, domains: tuple[str, ...], deadline: float) -> bytes:
public_https(url, domains)
request = Request(
url,
headers={"Accept": "image/png,image/jpeg,image/webp", "User-Agent": "gateway-image-example/1"},
method="GET",
)
if time.monotonic() >= deadline:
raise TimeoutError("asset deadline reached")
timeout = max(0.1, min(READ_SLICE_S, deadline - time.monotonic()))
try:
with HTTP.open(request, timeout=timeout) as response:
content_type = response.headers.get_content_type()
if content_type not in {"image/png", "image/jpeg", "image/webp"}:
raise RuntimeError("asset response is not an approved image MIME")
blob = bounded_read(response, IMAGE_CAP, deadline, None)
except HTTPError as exc:
error_hash = safe_error_hash(exc, deadline)
raise GatewayHTTPError(
exc.code, f"asset fetch failed with HTTP {exc.code}; body_sha256={error_hash}"
) from None
mime, _, _, _ = check_image(blob)
if content_type != mime:
raise RuntimeError("asset MIME/magic mismatch")
return blob
def atomic_bytes(path: Path, blob: bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
handle: int | None = None
temp_name: str | None = None
try:
handle, temp_name = tempfile.mkstemp(prefix=path.name + ".", dir=path.parent)
try:
stream = os.fdopen(handle, "wb")
handle = None
with stream:
stream.write(blob)
stream.flush()
os.fsync(stream.fileno())
except Exception:
if handle is not None:
try:
os.close(handle)
except OSError:
pass
handle = None
raise
os.replace(temp_name, path)
temp_name = None
finally:
if handle is not None:
try:
os.close(handle)
except OSError:
pass
if temp_name and os.path.exists(temp_name):
try:
os.unlink(temp_name)
except OSError:
pass
def atomic_json(path: Path, value: dict[str, Any]) -> None:
atomic_bytes(path, (json.dumps(value, indent=2, sort_keys=True) + "\n").encode())
def exclusive_json(path: Path, value: dict[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
if hasattr(os, "O_BINARY"):
flags |= os.O_BINARY
handle: int | None = None
created = False
try:
handle = os.open(path, flags, 0o600)
created = True
try:
stream = os.fdopen(handle, "wb")
handle = None
with stream:
stream.write((json.dumps(value, indent=2, sort_keys=True) + "\n").encode())
stream.flush()
os.fsync(stream.fileno())
except Exception:
if handle is not None:
try:
os.close(handle)
except OSError:
pass
handle = None
raise
except Exception:
if created:
try:
os.unlink(path)
except OSError:
pass
raise
finally:
if handle is not None:
try:
os.close(handle)
except OSError:
pass
def read_local_json(path: Path) -> dict[str, Any]:
raw = path.read_bytes()
if len(raw) > JSON_CAP:
raise RuntimeError("local ledger exceeds cap")
value = json.loads(raw)
if not isinstance(value, dict):
raise RuntimeError("local ledger is not an object")
return value
def update_ledger(path: Path, **changes: Any) -> dict[str, Any]:
current = read_local_json(path)
current.update(changes)
atomic_json(path, current)
return current
def append_ledger_event(path: Path, event: dict[str, Any]) -> None:
current = read_local_json(path)
events = current.get("events")
if not isinstance(events, list):
events = []
if len(events) >= 100:
events = events[-99:]
events.append({"at": utc_now(), **event})
current["events"] = events
current["updated_at"] = utc_now()
atomic_json(path, current)
def make_plan() -> Plan:
gateway = os.getenv("IMG_GATEWAY", "fal").lower()
prompt = os.getenv("IMAGE_PROMPT", "").strip()
if not prompt:
raise ValueError("IMAGE_PROMPT is required")
seed = int(os.getenv("IMAGE_SEED", "1"))
if gateway == "fal":
model = "fal-ai/flux/schnell"
return Plan(
gateway,
model,
"https://queue.fal.run/fal-ai/flux/schnell",
{
"prompt": prompt,
"image_size": "square_hd",
"num_inference_steps": 4,
"num_images": 1,
"seed": seed,
"enable_safety_checker": True,
"output_format": "png",
},
"queue",
"floating endpoint ID; schema snapshot required",
"https://fal.ai/docs/model-api-reference/image-generation-api/flux-schnell",
"https://api.fal.ai/v1/models/pricing?endpoint_id=fal-ai/flux/schnell",
"megapixel-rounded-up",
Decimal("0.003"),
Decimal("2"),
)
if gateway == "replicate":
model = "black-forest-labs/flux-schnell"
version = os.getenv("REPLICATE_VERSION_ID", "").strip().lower()
inputs = {
"prompt": prompt,
"aspect_ratio": "1:1",
"num_outputs": 1,
"num_inference_steps": 4,
"seed": seed,
"output_format": "webp",
"output_quality": 90,
"disable_safety_checker": False,
"go_fast": False,
"megapixels": "1",
}
if version:
require_sha256("REPLICATE_VERSION_ID", version)
return Plan(
gateway,
model,
"https://api.replicate.com/v1/predictions",
{"version": version, "input": inputs},
"prediction",
"immutable version requested and preflight-required",
f"https://api.replicate.com/v1/models/black-forest-labs/flux-schnell/versions/{version}",
"https://replicate.com/black-forest-labs/flux-schnell/api",
"output-image",
Decimal("0.003"),
Decimal("1"),
)
return Plan(
gateway,
model,
"https://api.replicate.com/v1/models/black-forest-labs/flux-schnell/predictions",
{"input": inputs},
"prediction",
"official model follows maintained latest; schema preflight-required",
"https://api.replicate.com/v1/models/black-forest-labs/flux-schnell",
"https://replicate.com/black-forest-labs/flux-schnell/api",
"output-image",
Decimal("0.003"),
Decimal("1"),
)
if gateway == "together":
model = "black-forest-labs/FLUX.1-schnell"
return Plan(
gateway,
model,
"https://api.together.ai/v1/images/generations",
{
"model": model,
"prompt": prompt,
"width": 1024,
"height": 1024,
"steps": 4,
"n": 1,
"seed": seed,
"response_format": "base64",
"output_format": "png",
"disable_safety_checker": False,
},
"synchronous",
"model ID may redirect; response identity enforced",
"https://docs.together.ai/reference/post-images-generations",
"https://www.together.ai/pricing",
"megapixel",
Decimal("0.0027"),
Decimal("1.048576"),
)
raise ValueError("IMG_GATEWAY must be fal, replicate, or together")
def merge_schema(schema: dict[str, Any]) -> dict[str, Any]:
if "allOf" not in schema:
return schema
merged: dict[str, Any] = {"properties": {}, "required": []}
for part in schema.get("allOf", []):
if isinstance(part, dict):
part = merge_schema(part)
merged["properties"].update(part.get("properties", {}))
merged["required"].extend(part.get("required", []))
return merged
def replicate_input_schema(openapi: dict[str, Any]) -> dict[str, Any]:
schemas = openapi.get("components", {}).get("schemas", {})
candidate = schemas.get("Input") if isinstance(schemas, dict) else None
if not isinstance(candidate, dict):
raise RuntimeError("Replicate OpenAPI has no Input schema")
candidate = merge_schema(candidate)
if not isinstance(candidate.get("properties"), dict):
raise RuntimeError("Replicate Input schema has no properties")
return candidate
def validate_schema_value(name: str, value: Any, schema: dict[str, Any]) -> None:
expected = schema.get("type")
valid = {
"string": isinstance(value, str),
"integer": isinstance(value, int) and not isinstance(value, bool),
"number": isinstance(value, (int, float)) and not isinstance(value, bool),
"boolean": isinstance(value, bool),
"array": isinstance(value, list),
"object": isinstance(value, dict),
}
if expected in valid and not valid[expected]:
raise RuntimeError(f"Replicate input {name!r} violates schema type")
if "enum" in schema and value not in schema["enum"]:
raise RuntimeError(f"Replicate input {name!r} violates schema enum")
if isinstance(value, (int, float)) and not isinstance(value, bool):
if "minimum" in schema and value < schema["minimum"]:
raise RuntimeError(f"Replicate input {name!r} is below schema minimum")
if "maximum" in schema and value > schema["maximum"]:
raise RuntimeError(f"Replicate input {name!r} exceeds schema maximum")
def validate_replicate_inputs(openapi: dict[str, Any], inputs: dict[str, Any]) -> None:
schema = replicate_input_schema(openapi)
properties = schema["properties"]
unknown = sorted(set(inputs) - set(properties))
missing = sorted(set(schema.get("required", [])) - set(inputs))
if unknown or missing:
raise RuntimeError(f"Replicate schema mismatch; unknown={unknown}; missing={missing}")
for name, value in inputs.items():
field = properties[name]
if not isinstance(field, dict):
raise RuntimeError(f"Replicate input schema for {name!r} is invalid")
validate_schema_value(name, value, field)
def preflight_replicate(plan: Plan, token: str, expected_hash: str) -> dict[str, Any]:
deadline = time.monotonic() + CREATE_DEADLINE_S
auth = "Bearer " + token
info = get_json(plan.schema_url, "api.replicate.com", "/v1/models/black-forest-labs/flux-schnell", auth, deadline)
requested_version = plan.payload.get("version")
if requested_version:
if info.get("id") != requested_version:
raise RuntimeError("Replicate version preflight did not return the requested version")
version = requested_version
openapi = info.get("openapi_schema")
else:
if info.get("owner") not in (None, "black-forest-labs") or info.get("name") not in (
None,
"flux-schnell",
):
raise RuntimeError("Replicate model preflight returned a different model")
latest = info.get("latest_version")
if not isinstance(latest, dict) or not isinstance(latest.get("id"), str):
raise RuntimeError("Replicate model preflight has no latest version")
version = latest["id"]
openapi = latest.get("openapi_schema")
if not isinstance(openapi, dict):
raise RuntimeError("Replicate preflight has no OpenAPI schema")
actual_hash = hashlib.sha256(canonical(openapi)).hexdigest()
if actual_hash != expected_hash:
raise RuntimeError("Replicate schema hash drifted; do not create")
validate_replicate_inputs(openapi, plan.payload["input"])
return {
"url": plan.schema_url,
"sha256": actual_hash,
"checked_at": utc_now(),
"version": version,
"validation": "model/version identity and Input allowlist passed before create",
}
def wait_fal(
plan: Plan,
token: str,
on_identity: Callable[[str, dict[str, Any]], None],
on_state: Callable[[str], None],
on_retry: Callable[[dict[str, Any]], None],
resume_id: str | None = None,
) -> tuple[str, dict[str, Any], list[bytes]]:
auth = "Key " + token
deadline = time.monotonic() + POLL_DEADLINE_S
if resume_id:
request_id = valid_provider_id(resume_id, "fal")
else:
initial = post_json(
plan.url,
"queue.fal.run",
"/fal-ai/flux/schnell",
auth,
plan.payload,
{
"X-Fal-Store-IO": "0",
"X-Fal-No-Retry": "1",
"x-app-fal-disable-fallback": "1",
"X-Fal-Object-Lifecycle-Preference": json.dumps(
{"expiration_duration_seconds": 3600}
),
},
)
request_id = valid_provider_id(initial.get("request_id"), "fal")
on_identity(request_id, initial)
root = f"https://queue.fal.run/{plan.model}/requests/{request_id}"
while True:
state = get_json(
root + "/status",
"queue.fal.run",
f"/{plan.model}/reque
…(truncated)