qualcomm Model Builder
How to use this SKILL (it is a thin dispatch layer):
- Pass the Boundary Decision gate below first — it decides whether this skill even applies.
- Use the Routing Table to load the ONE reference / sub-SKILL that matches your step or problem — do NOT read everything up front.
- Follow the Core Workflow spine; open
references/core_workflow.mdfor per-step commands.- The Blocking Conditions and Disciplines below are the only rules you must hold in mind the whole time.
- Trust the docs: never run commands to re-verify facts already in this file or
${APP_ROOT}\data\config\qairt_env.json(torch/Python versions, tool paths). Don't read script source unless a reference doc is missing the detail (then update that doc).- x64 host + user wants to run inference locally on this machine? Read
${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.mdFIRST (before Step 7). If HOST_OS is ARM64, or if inference will go via ADB, skip that file entirely.
🚨 Boundary Decision — pass BEFORE activating (MANDATORY GATE)
Answer three questions; if any veto holds, stop and switch skills.
| # | Question | YES -> | NO -> |
|---|---|---|---|
| Q1 | Does the model already have a prebuilt package on AI Hub (Zipformer, MobileNet, YOLO, …)? | ❌ Stop -> model-hub skill |
Q2 |
| Q2 | Is the file to download/use an AI Hub prebuilt artifact (QNN context binary .bin / .dlc)? |
❌ Stop -> model-hub skill |
Q3 |
| Q3 | Does the user have a custom ONNX/PyTorch model to convert, or to re-quantize/recompile into a custom .bin? |
✅ Activate this skill | ❓ Intent unclear — confirm first |
Switch to model-hub on any of: "download from AI Hub" / "model on AI Hub" / "prebuilt package" / "on-device pre-exported package"; a model name + "download" where it already exists on AI Hub; "QNN_CONTEXT_BINARY" / "QNN_DLC". An AI Hub prebuilt package (.bin/.dlc) is loaded directly by model-hub via qai_appbuilder/QNNContext; this skill only converts/compiles custom ONNX/PyTorch. ("run inference" alone does NOT pick this skill — only when the target is a self-converted model.)
Dispatching a sub-agent to another skill (e.g.
model-hub) — MANDATORY: a sub-agent inherits NOTHING from this skill (blank context). Once a vacuum-written prompt sent one recursively scanningC:\/C:\WoS_AIfor.bin-> 30-min hang. So: (1) **firstreadthe target SKILL.md in full**, write the prompt from it; (2) the prompt's first instruction MUST be "read the target SKILL.md in full before acting"; (3) the prompt MUST NOT carry any path/script/toolchain from this skill (run_pipeline.py,qnn-onnx-converter, …) — useless for prebuilt packages and lures a wrong disk scan; (4) reuse the target's Sub-Agent Dispatch Template if it has one.
⚠️ First action after activating (before Step 1): ask the DLC portability question (cross-platform vs SoC-optimised DLC) unless the user's request already signals it. Full wording + CLI mapping + skip-signals ->
references/operations_reference.md § DLC Portability Question.
🧭 Routing Table — load only the ONE that matches (MANDATORY)
This SKILL is the main pipeline + index. For any specific step, error, or topic, load the single matching doc below — do NOT read them all. Troubleshooting sub-SKILLs are the canonical authority for errors (more complete than any inline summary). base ships in the external edition; advanced is internal-only (silently skip if the file is absent).
By problem / error (troubleshooting sub-SKILLs — canonical):
| Symptom / error code | Load | tier |
|---|---|---|
unsupported operator / 0xc26 / Einsum / Mod / Floor / ScatterND / dry-run false positive |
${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/operator-patching/SKILL.md |
base |
Graph Compose failure / graph_names / Wrong number of Parameters 5 / loadRemoteSymbols 4000 / arch mismatch |
${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/conversion-troubleshooting/SKILL.md |
base |
| QNNContext crash / stale artifact / multi-model same-process / Linux HTP transport mismatch / NCHW-NHWC wrong | ${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/inference-troubleshooting/SKILL.md |
base |
VCTargetsPath / CMake / import cv2·Pillow / qai_appbuilder import fails |
${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/env-troubleshooting/SKILL.md |
base |
0-byte generator / WinError 193 / need to modify an SDK file |
${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/sdk-integrity-recovery/SKILL.md |
base |
| basicsr / functional_tensor / aux-branch ReshapeOp (ONNX export) | ${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/export-troubleshooting/SKILL.md |
base |
By topic (references):
| Topic | Load |
|---|---|
| Core Workflow step details (commands + caveats) | references/core_workflow.md |
| Operations detail (flow selection, DLC-portability Q, guardrails, working-dir, project config, script index, pack export) | references/operations_reference.md |
| Environment setup (Windows) | references/win_qairt_setup.md |
| Export + ONNX validation | references/model_export_validation.md |
| Operator patching (full code library) | references/operator_patching.md |
| QNN conversion | references/qnn_conversion.md |
| SNPE conversion | references/snpe_conversion.md |
| Quantization (+ tool-param map) | references/model_quantization.md |
| Context binary | references/context_binary.md |
| Inference (NCHW/NHWC, API, templates) | references/inference.md |
| QNN inference routing (per-platform defaults + override keywords) | ${APP_ROOT}/factory/chat_features/_shared/qnn-inference-routing.md |
| x64 host — local inference guide (opt-in; compatibility matrix, backend choice via question tool, B11, closing statement) | ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md |
| Quantization sensitivity (pre-conversion risk pre-flight) | references/quantization-sensitivity.md |
| Verification discipline | references/verification-discipline.md |
Pack export & inference_manifest.json |
references/pack_export.md |
| ADB device deployment | references/adb_execution.md |
| Remote (SSH) execution | references/remote_execution.md |
| Troubleshooting quick-index + Windows tips | references/troubleshooting.md |
Core Workflow (8-step spine)
Per-step commands, caveats, and MANDATORY sub-requirements ->
references/core_workflow.md— open it when you start executing. First (once, before Step 1): run Host OS Detection, writeHOST_OStoplan.md(windows-arm64/windows-x64/linux-aarch64/linux-x64; drives Step 3 backend + Step 7 path) ->core_workflow.md § Host OS Detection.
- Export to ONNX —
python_x64_venv,model.eval(), FP32 only (never FP16),opset_version=18; disable training-only branches (aux_logits/dropout). ->core_workflow.md § Step 1/model_export_validation.md. - Inspect ONNX I/O —
qai_inspect_onnxio.py. ⚠️ Do NOT gate on--dry_run(false positives) — go straight to Step 4. ->core_workflow.md § Step 2. - Operator patching — ONLY if actual conversion hits a hard op error. Patch in-memory, re-validate (checker -> real conversion -> cosine ≥ 0.95). Canonical ->
operator-patchingsub-SKILL. - Convert float model —
run_pipeline.py(Flow A, default, all hosts).run_pipeline_legacy.py/qai_convert_fp.py/qai_convert_int.pyare Flow C (DLL,windows-arm64only; error out elsewhere).--precision fp16|fp32. Do NOT manually pass--htp_version—run_pipeline.pyauto-detects on Linux viaqnn-platform-validator; Windows defaults to v73. Only specify manually when auto-detection fails AND you know the target SoC (seecore_workflow.md § Step 4for the full SoC→HTP table). ->core_workflow.md § Step 4/qnn_conversion.md. - Quantization (optional) —
run_pipeline.py --precision <p> --calib_list <list>. ⚠️ Real multi-class calibration data; ask user if none. ->core_workflow.md § Step 5/model_quantization.md. - Context binary —
run_pipeline.pyemits.binautomatically. The.binis a QNN context binary for the HTP backend; on ARM64 hosts it targets real HTP. For loading a.binon an x64 host, seex64-host-notes.md. Portable across HTP backend builds (routing doc §4); use.dlcfor cross-backend / late backend choice. ->core_workflow.md § Step 6/context_binary.md. - Inference + validation — route by
HOST_OSper${APP_ROOT}/factory/chat_features/_shared/qnn-inference-routing.md(the routing doc): ARM64 hosts default to local HTP via Path A (qai_runner.py+qai_appbuilder,python_runtime_venv); x64 hosts default to ADB via Path B (adb_runner.py), with opt-in local execution when the user asks — see${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md. User override keywords can flip the default (routing doc §3). NEVER callqnn-net-rundirectly. MANDATORY: saveinfer_{MODEL}.py+inference_manifest.json; any x64-local-execution run must emit the closing statement (seex64-host-notes.md§4) in the user's language. Per-step details ->core_workflow.md § Step 7/inference.md. - Validation report (MANDATORY) — ONNX (CPU-only) vs QNN cosine (≥0.99 FP16/FP32, ≥0.95 INT); below threshold -> B6 (stop, don't auto-fix). Write
REPORT.mdwith the exact "Cosine Similarity Summary" plain-text format. Print${WORKSPACE}\<model_name>in every turn's final summary. Updateplan.md. ->core_workflow.md § Step 8.
Artifact checklist per Flow (A/B/C) ->
references/expected_output_artifacts.md. Batch runs ->scripts/model_config.json.
🛑 Blocking Conditions (always STOP & ask — both modes)
| # | Condition -> Action |
|---|---|
| B1 | Required config var empty/placeholder -> stop, list missing, ask user. |
| B2 | pip install needed -> stop, state package + reason, ask permission. |
| B3 | Patch iterations exhausted, NO progress (same ops, no patterns left) -> stop, list attempts + logs, escalate. |
| B4 | Operator patch would change model semantics -> stop, describe change, ask approval. |
| B5 | Target device unavailable for context-bin gen / on-device test (incl. remote unreachable) -> stop, ask how to proceed. |
| B6 | Accuracy < threshold after quant (cosine < 0.95) -> do NOT auto-fix. ① zero-cost diagnosis (is calibration one image / its augmentations? not diverse). ② STOP, report cosine + diagnosis, present options (each 1-line principle), ask which: (1) improve calib diversity; (2) --cle (+--per_channel); (3) --precision w8a16; (4) keep FP16 / try bf16; (5) accept if Top-K correct. Full flow -> model_quantization.md / quant-accuracy sub-SKILL. |
| B7 | No known replacement pattern for an unsupported operator -> stop, document, escalate. |
| B8 | Context binary gen fails on windows-arm64 / linux-aarch64 (real HTP hosts) -> stop (run_pipeline.py exits non-zero; NOT silently degraded). Return to operator patching; do NOT retry alternate generators (x86_64 build can't load an ARM64 DLL). 0-byte/corrupt generator = damaged SDK file -> qai_dev_gen_contextbin.py self-heals from the kept SDK zip; if none -> sdk-integrity-recovery sub-SKILL. Diagnose READ-ONLY. On x64 hosts see ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md for loading the .bin locally and applicable blocking rules (B11). |
| B9 | Fixing would require modifying any file under $QAIRT_SDK_ROOT/$QNN_SDK_ROOT -> STOP IMMEDIATELY. Never edit/copy-over/rename/delete an SDK file (the C:\Qualcomm tree is tool-layer write-protected). Copy the file into the workspace and edit the copy, pointing tooling at it via documented overrides (--config_file, QNN_* env, workspace-local backend_extensions.json). Reading the SDK dir is fine. Genuinely missing/corrupt -> recover from kept zip (sdk-integrity-recovery); ask explicitly "edit <sdk_path>/<file>? [y/N]" and act only on a scoped yes naming the file. |
| B10 | A tool/script/package not described here must run and the venv is unclear -> stop, ask. Default to python_x64_venv for conversion tools (python310.dll); use python_runtime_venv (aka legacy python_arm64_venv, resolves to .venv_arm64_313 on WoS / .venv_x64_313 on x64) only for qai_appbuilder/QNNContext inference. Still unsure -> ask. |
| B11 | Any x64 local execution run (see ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md) — user asks to report the numbers as real HTP performance → stop. Full definition, options to present, and closing statement → x64-host-notes.md §5 & §4. |
Disciplines (hold these the whole run)
- Execution mode (
MODEin config, defaultbatch): batch = run all phases autonomously, apply safe defaults, log decisions, only stop on a Blocking Condition — do NOT ask "proceed to next phase?" / "which precision?" (use config) / "run onnxsim?" (always).interactive= confirm at each phase. Never silently fall back to ONNX/CPU when QNN/HTP fails — diagnose & fix, or stop & report; substituting CPU for a failed HTP run is never an acceptable fix. - Inference results MUST come from actual execution. Never output Top-K/confidence/latency/cosine without first running the script via
exec; every number traces to anexeclog line. No guessing/estimating from model knowledge; no writing the report before running. - Operator patching is exhaustive — patch ALL unsupported ops until no pattern remains; never fall back to CPU; no fixed iteration cap; escalate only on B7/B4/B3. Rules + code ->
references/operator_patching.md. - Working directory: all model artifacts under
${WORKSPACE}\<model_name>\— NEVER under aQAIModelBuilderpath, home/Downloads, or a CWD outside${WORKSPACE}. Self-check every write. Bootstrap withqai_workspace_init.py. Tables + init diagnosis ->references/operations_reference.md § Working Directory. - Wrappers only: conversion via
run_pipeline.py(run_pipeline_legacy.py= Flow C,windows-arm64only); inference viaqai_runner.py/qai_appbuilder(neverqnn-net-run). Wrappers handle--preserve_io, layout, PYTHONPATH, arch dirs, and host_arch routing. - Trust known facts; never re-verify via shell (torch=2.x, Python x64=3.10/ARM64=3.13, tool paths — all in
qairt_env.json). Timeouts:timeout=0for all conversion commands. Benign HTP errors,os._exitcrash, encoding, escalation, SDK read-only rules ->references/operations_reference.md § Guardrails.
⚠️ Python Environments (not interchangeable — gates B10)
Paths from ${APP_ROOT}\data\config\qairt_env.json (Setup.bat generates it). Never hardcode.
| Env | Key | Python | Role |
|---|---|---|---|
| Conversion | python_x64_venv |
x86_64 3.10 | ONNX export, qairt-converter, qairt-quantizer, qnn-onnx-converter, qnn-model-lib-generator (all hosts). |
| Runtime | python_runtime_venv (fallback: python_arm64_venv) |
3.13 — aarch64 on WoS (.venv_arm64_313), x86_64 on x64 Windows (.venv_x64_313) |
qai_appbuilder, QNNContext, inference. |
| Ubuntu | python3_venv |
x86_64 3.12 | All Ubuntu ops (no ARM64 venv on Ubuntu). |
Default for tools not listed here: python_x64_venv (most QAIRT tools link python310.dll); switch to python_runtime_venv only when the tool imports qai_appbuilder/QNNContext or runs inference on a .bin/.dlc; unsure -> B10. On x64 hosts, if the user asks to run inference locally, use python_runtime_venv (resolves to .venv_x64_313) and see ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md for backend selection. Setup / pip / --index-url / opencv / PYTHONPATH -> references/win_qairt_setup.md; env broken -> env-troubleshooting sub-SKILL.