Forge Multimodal Router

forge-multimodal-router

Purpose

The federation supports four modalities (vision, audio, video, somatic) across 8 agents. Each modality has multiple functions (recognize, generate, edit, clone, analyze) and multiple engines (image gen, video gen, TTS, ASR, voice clone, music). Without a single routing skill, decisions are scattered across ~18 skills (token-plan-image, token-plan-speech, token-plan-video, AAA-tts-engine-catalog, minimax-image-gen, creative/minimax-cli, etc.) and 3 knowledge graphs.

This skill is the single ingestion entry point for any multimodal routing question. It does NOT duplicate model identity — it references the SOT.

The authority doctrine (DO NOT VIOLATE)

Rule: Model identity lives ONLY in SOT. Skill/tool surface (Hermes config, OpenCode config, etc.) is a CONSUMER, not authority. Next model release = 1 SOT edit (RCR — Release Change Ratio).

Picker sync rule: AAA/FED has authority to write SOT (sqlite + json) but does NOT auto-write to picker configs (Kimi/Codex/Qwen/Claude/Go/Grok CLI). F13 holds the pen. Manual alignment audit is correct path.

The routing decision algorithm

Given a multimodal request, ask in order:

1. What modality? (vision / audio / video / somatic)
2. What direction? (input = perception / output = generation)
3. What function? (recognize / generate / edit / clone / analyze / transform)
4. What constraints? (real-time? free quota required? Malaysian voice? identity-preserving? i-ARIF voice?)

Then resolve:

resolution = lookup_sot(
  modality, direction, function, constraints
)
# SOT returns: { provider_id, model_key, endpoint_url, params_override }

If SOT doesn't have a matching entry → STOP. Do NOT fallback to agent card or pickers. Either:

• Check arifos-federation-provider-multimodal-discovery skill to probe the provider
• Or forge an ephemeral skill for the new routing case

Routing rules (read from SOT, this table is a navigation aid ONLY)

Vision

Audio

Video

Somatic (Music Intelligence)

Constraints & override rules

Free quota preference

• If SOT model is in Singapore region AND has International scope AND has blue quota bar in console → prefer it over PAYG
• Validate via arifos-federation-provider-multimodal-discovery skill before assuming free quota
• Enable Free Quota Only mode per model in console (default OFF for verified users)

i-ARIF voice identity

• F13 SOVEREIGN-gated: voice identity is biometric-class
• Use AAA-voice-cloning-mimo-minimax for any i-ARIF request
• All i-ARIF calls seal to VAULT999 with explicit voice_id annotation

Identity preservation (face/body)

• F13, 2026-08-12: "Hang jangan ubah muka manusia"
• Real photo edits MUST preserve face, body, skin tone 100%
• Use media/aaa-image-editing (6 Iron Rules) + ensemble NB2 + NB-Pro for identity-critical work

Privacy / Sovereignty

• Voice biometric = F1 custody (vault-class)
• Face ID = local-only (no cloud round-trip)
• All auth keys via ~/.secrets/kunci-root.env (5-R Protocol)

How to invoke this skill

When an agent receives a multimodal request and needs to route:

1. Classify the modality + direction + function + constraints
2. Look up the SOT key (use this skill's routing table as index, then verify against /root/.config/federation-models.json)
3. Verify the SOT entry is current (F11: log the lookup with SHA256 of SOT file)
4. Resolve to provider_id + model_key + endpoint_url
5. Call the model via the agent's MCP surface (e.g., mcp__aforge__forge_browser_* for browser, or direct API call for non-MCP)
6. If the route is Grok Imagine: spawn /root/.grok/bin/grok-multimodal.sh. Those tools are harness-native to FI-007, not Hermes/OpenCode.
7. Log the routing decision to VAULT999 (F11 AUDIT)

Harness-native Imagine (not a FED model)

Telephone: /root/AAA/docs/GROK_IMAGINE.md. Wrapper: grok-multimodal. Do not add grok to LiteLLM.

Cross-cutting doctrine (apply across modalities)

Knowledge graphs (canonical unified maps)

Floors

• F2 TRUTH: Every routing decision cites the SOT entry used (provider_id + model_key + SHA256 of SOT file at lookup time).
• F4 CLARITY: This skill is a navigation index, not a duplicate of SOT. Single source of truth = SOT.
• F8 GENIUS: Cross-modal requests route through η algorithm (4 questions) before falling back to per-modality skills.
• F11 AUDIT: Every routing decision logged to VAULT999 receipts (provider + model + token estimate + SOT version).
• F1 AMANAH: Free quota preference + BlueQuotaBar check before PAYG. No surprise charges.

Annual verification

This skill MUST be re-validated:

• On any SOT change to /root/.config/federation-models.json (RCR enforcement)
• On any new model release touching vision/audio/video/somatic
• Quarterly (next due: 2026-11-20)

If the routing table here drifts from SOT, the table is wrong (SOT wins). Update this skill to match SOT, never the reverse.

Failure modes

Audit trail

• 2026-08-25 — FI-007: Imagine spawn path for all AAA agents (grok-multimodal.sh + GROK_IMAGINE.md). Harness-native, not FED.
• 2026-08-20 — forged by FI-003 Qwen Code
• Driver: Arif F13 directive "now make sure my hermes agent telegram ASI_arifos_bot know how to use this"
• Builds on: 32 multimodal models wired into /root/.hermes/config.yaml dashscope-payg provider + tts/stt.qwen-audio-payg entries (2026-08-20)
• Builds on: 3 canonical knowledge graphs (audio + visual + video) — /root/AAA/knowledge-graph/
• Constraints: F2 SOT authority, F11 AUDIT, F13 picker sovereignty, RCR 1-edit-per-release
• Receipt: to be appended to /root/forge_work/qwen-sessions/sessions.jsonl + named receipt