LiteRT on Google Tensor: compile, benchmark, package for apps
Three scriptable workflows for Pixel TPUs:
- Compile a plain
.tflite into a Google Tensor AOT-compiled .tflite
with the public LiteRT Python API (ai_edge_litert.aot), locally on Linux
x86_64 or on a Google Colab VM (--colab, via google-colab-cli) when
the user is on macOS/Windows or lacks a Linux box.
- Benchmark a compiled model on a connected Pixel with LiteRT's
benchmark_model, NPU-only, optionally next to a CPU baseline, including
TPU tile time (pure TPU compute time) and TPU energy per inference.
- Package compiled builds for an Android (Gradle) app as a Google Play AI
pack with the Google Tensor NPU runtime and a Kotlin loader (or, for an app
that already ships Google Tensor builds, only the files to replace), apply
the plan to the project when the user agrees, and check the app on a Pixel.
Everything runs synchronously in the user's terminal; the scripts never start
background processes and always clean up what they pushed to the phone or
allocated on Colab. All paths below are relative to this skill's directory
(SKILL_DIR).
| Path |
Purpose |
| assets/versions.json |
LiteRT versions the scripts use (single source) |
| scripts/check_compiled.py |
Is this .tflite already Tensor-compiled? |
| scripts/compile_model.py |
AOT compile (local Linux x86_64 or --colab) |
| scripts/benchmark_on_device.py |
adb push -> benchmark -> parse -> cleanup |
| scripts/package_ai_pack.py |
Compiled builds -> AI pack + Gradle modules for an app, or an update of one |
| scripts/install_and_verify.py |
Build the app, install it like Google Play, read the NPU/CPU result |
| assets/templates/ |
Gradle, Kotlin, INTEGRATION.md and UPDATE.md templates (package_ai_pack.py) |
| references/compile-flags.md |
API, compiler flags, script options, --colab |
| references/benchmark-flags.md |
What the benchmark runs, binaries, writing the report, example output |
| references/app-integration.md |
Packaging for an app: contents, checks, delivery modes, applying the plan, checking on a Pixel |
| references/troubleshooting.md |
Symptom -> fix tables |
| references/contributing.md |
Maintainers: adopting a new LiteRT release |
Every script supports --help and --dry-run and is Python 3 standard library
only, so it runs with any python3, with two exceptions that import LiteRT: a
local compile (compile_model.py without --colab, --dry-run or
--print-install) must use the virtualenv's interpreter
(<venv>/bin/python SKILL_DIR/scripts/compile_model.py ..., or activate the
venv first), and a real package_ai_pack.py run (not --dry-run or
--check-project) needs a Python with ai-edge-litert (the compile venv
works; the Beta SDK is not needed).
Prerequisites (tell the user up front)
| Need |
Compile (local) |
Compile (--colab) |
Benchmark |
Package (app) |
Google Tensor SDK Beta access - sign up at https://services.google.com/fb/forms/tensor_ml_sdk_experimental_access (overview: https://developers.google.com/edge/litert/next/tensor-sdk). Google emails a download link for litert_plugin_compiler.tar.gz; the compiler cannot be installed without it. |
required |
required (--sdk-tarball or --sdk-url) |
- |
- |
| Host OS |
Linux x86_64, >= 16 GB RAM |
any OS (colab CLI on PATH; runs on a remote Colab Linux x86_64 VM) |
any OS with adb |
any OS where ai-edge-litert installs (the compile venv's Linux x86_64 host works) |
| Python environment |
Local venv with ai-edge-litert + ai-edge-litert-sdk-google-tensor at the version in versions.json (compile_model.py --print-install) |
Any local python3 + google-colab-cli (compile_model.py --print-install --colab); SDK is installed on the Colab VM automatically |
any python3 (stdlib only) |
a Python with ai-edge-litert (the compile venv works; the SDK package is not needed) |
adb on PATH, Pixel 10/11 with USB debugging enabled |
- |
- |
required |
step 7 (install and check the app) |
A JDK the app's Gradle supports (JAVA_HOME), the Android SDK, the project's ./gradlew |
- |
- |
- |
steps 6-7 (build, install); bundletool is downloaded once |
libLiteRtDispatch_GoogleTensor.so - downloaded automatically from the public LiteRT binaries bucket (same directory as benchmark_model), or supplied with --dispatch-lib |
- |
- |
required |
- (the app gets it from Google Maven) |
| Internet |
once for pip install |
required (Colab VM + pip install) |
once to fetch benchmark_model + dispatch lib (cached in ~/.cache/litert-google-tensor/) unless supplied locally |
- (the app's Gradle build downloads the LiteRT Maven artifacts) |
Supported targets: Tensor_G5 (Pixel 10 family) and Tensor_G6 (Pixel 11
family), as listed by the public Tensor SDK docs; Pixel 9 and older are not.
Step 0 - Collect parameters
Ask the user only for what is missing; never invent values:
| Parameter |
How to get it |
| Model path |
Absolute path to the .tflite. Confirm it exists. |
| Goal |
check only, compile only, benchmark only, compile + benchmark, or package for an app (step 5). Infer it from the request ("how fast is it on the TPU vs CPU" = compile if needed + benchmark with a CPU baseline; "put it in my app" / "deploy it" = package); do not add an unrequested device run. |
| Target SoC |
For a compile, the user's stated device wins (Pixel 10 family = Tensor_G5, Pixel 11 family = Tensor_G6), even if a different Pixel is attached. For a benchmark, the attached phone decides: adb shell getprop ro.soc.model -> Tensor G5 => Tensor_G5, Tensor G6 => Tensor_G6 (the benchmark script reads and enforces this itself). |
| Output path |
Default <output-dir>/<stem>_Google_<target>.tflite. compile_model.py refuses to overwrite an existing file (exit 2): run check_compiled.py on the existing file and reuse it, or pass --output-name, or --overwrite after asking. When a second build is needed next to an existing one, name it <stem>_Google_<target>.<YYYYMMDD-HHMM>.tflite (e.g. --output-name model_Google_Tensor_G5.20260107-0930.tflite) or, for a flag variant, <stem>_Google_<target>.<flag-summary>.tflite (...G5.half-maxsharding.tflite); keep the _Google_<target> part so the file stays recognisable. |
| Device serial |
Only if adb devices lists more than one device. |
| Dispatch library / benchmark binary |
Optional. Both are downloaded automatically from the same public bucket directory and cached. Only ask if the user wants a specific build, or if the host is offline: then --benchmark-bin <benchmark_model> plus --dispatch-lib <libLiteRtDispatch_GoogleTensor.so | dir | .zip | .aar> (also env LITERT_GOOGLE_TENSOR_DISPATCH_LIB) use local copies and no network is touched. |
| CPU baseline |
Optional; needs the path of the original uncompiled model (ask for it if only a compiled file was given; the benchmark script rejects a compiled file here with exit 2). |
| Run counts / mode |
Defaults 5 warmup / 50 measured, burst performance mode. benchmark_model treats counts as minimums and also runs at least 0.5 s warmup / 1 s measured, so fast models execute more runs than requested. If the user states exact counts, say so and ask whether to disable the floors (--extra-arg=--min_secs=0 --extra-arg=--warmup_min_secs=0); "steady-state" / "sustained" requests map to --performance-mode sustained_performance. |
| TPU tile time |
On by default, nothing to ask: a second, hooked NPU pass. --no-tile-time turns it off; the hooked pass itself (it repeats the NPU run, so slow models take about twice as long) is skipped only with --no-tile-time --no-power. |
| TPU energy |
On by default, nothing to ask: the power metric of the same hooked pass (the TPU rail's energy per inference). --no-power skips it. The script leaves it off, and says why, for a dispatch library whose power hook predates the fix. |
| Compile mode |
Local Linux x86_64 by default. Use --colab when the host is not Linux x86_64 (macOS/Windows/ARM) or when the user asks to compile on Google Colab; ask for the Beta tarball path (--sdk-tarball / GOOGLE_TENSOR_SDK_BETA) or URL (--sdk-url / GOOGLE_TENSOR_ML_SDK_URL) before running. |
| Compiler flags |
Default to the compiler defaults; only set flags the user asks for or that troubleshooting requires (see references/compile-flags.md). |
| App deployment level |
Packaging jobs only; nothing to ask up front. After step 5, tell the user the plan is ready (INTEGRATION.md, or UPDATE.md for an update) and ask whether you should apply it to their project (step 6). After that, offer to install the app on their Pixel and check it uses the NPU (step 7). |
| New integration or update |
Packaging jobs only. Ask whether the app already ships Google Tensor builds (an AI pack with folders like model#group_Google_Tensor_G5 under its src/main/assets). If it does, it is an update (step 5b): ask for the project root (the folder with settings.gradle.kts or settings.gradle). The pack, model file, device groups, delivery mode and app module then come from the project, so do not ask for them. |
| Builds to package |
One compiled build per target SoC (Tensor_G5, Tensor_G6) plus the original uncompiled model they were compiled from (the app's CPU path). Compile missing targets first (step 2); a build is per SoC, so a Pixel 11 needs a Tensor_G6 build. For an update, build every SoC the app already ships. |
| Delivery mode |
New integrations only (an update keeps the app's). When Google Play puts the model on the phone: install-time (default; installed with the app, ready at first launch), fast-follow (downloaded right after install) or on-demand (downloaded when the app asks). Explain each in one sentence (references/app-integration.md) and use the default when the user has no preference. |
| Pack / model names, app module |
New integrations only. Defaults: pack name = the original's file name made valid, model name = pack name, app module app, Kotlin package com.example.litert. Ask only when the user's project differs (another app module name, their package). |
| Quick test (optional) |
New integrations only. Offer it when the user wants to try the model on their own Pixel before setting up bundletool: --quick-test adds a copy of each build and the steps for ./gradlew installDebug -PtensorQuickTest=<SoC>, which puts the build and the NPU runtime in the debug APK. Local testing only; release builds are unaffected. |
Commands or a plan only (the user says not to run anything)
Run nothing - not even the read-only scripts - and give the same steps as a
real run, in the same order, as commands the user can paste:
- Start with
check_compiled.py on every input model (step 1), before
any install or compile command; checking only the compiled output
afterwards is not enough.
- For installs, give
python3 SKILL_DIR/scripts/compile_model.py --print-install [--colab] as the command that produces them (step 2a),
then the lines it prints with the user's paths filled in (venv,
export GOOGLE_TENSOR_SDK_BETA=<their tarball> before pip install) and
each version written as <version printed by --print-install>. Never
copy a version number from versions.json or from memory.
- Give no results, not even estimates: latency, speed-up, memory and ops
offloaded come only from the scripts' output. Say where they will appear
(for a benchmark, the table
benchmark_on_device.py prints and saves as
report.md).
- When the plan changes the user's app project (step 6), say that nothing
in the project changes until they approve, and end by asking whether you
should apply it.
Step 1 - Check whether the model is already compiled
python3 SKILL_DIR/scripts/check_compiled.py /abs/path/model.tflite [--expect-target Tensor_G5]
Run it on every model file involved (the model to compile or benchmark, the
CPU-baseline file, and an existing output file you are about to reuse). For a
benchmark job read the phone first (adb shell getprop ro.soc.model prints
Tensor G5 or Tensor G6) and pass the underscore form as --expect-target
(Tensor G5 -> --expect-target Tensor_G5), so one check answers both
"compiled?" and "for this phone?" (a plain model exits 1 either way); the
benchmark script enforces the match again anyway (exit 4).
| Exit |
Meaning |
Next action |
| 0 |
compiled for Google Tensor (SoC printed) |
skip compile; report the SoC (and the Compile info line if a .compile.json sidecar exists). Continue to step 3 only if the goal includes a benchmark. No sidecar (compiled elsewhere, flags unknown): for a plain "how fast is it" job, benchmark the file as-is and say "compile flags unknown"; when the user needs to know how it was built (or wants a reproducible comparison), compile the original with the defaults under a new --output-name and compare SHA-256 (sha256sum of both files, or output_sha256 in the new sidecar): compilation is deterministic, so an identical hash proves the existing file is a default build (references/compile-flags.md). Never --overwrite to find out. |
| 1 |
plain (uncompiled) model |
go to step 2 if compiling; this is the correct file for --cpu-baseline |
| 2 |
compiled for a different SoC than --expect-target |
compiled models are SoC-specific (a Tensor_G5 file never runs on a Tensor_G6 TPU, and a Tensor_G6 file never runs on a Tensor_G5 TPU; there is no runtime fallback, and --allow-mismatch only produces dispatch errors). Tell the user which SoC the phone has, then offer, in this order: an existing *_Google_<phone target>.tflite build of the same model in the project (verify it with this check first; do not benchmark it unless the user says so), attaching the matching phone, or recompiling the original uncompiled export for the phone's target. |
| 3 |
not a .tflite / missing |
fix the path |
| 4 |
compiled by another toolchain: an Edge TPU custom op (edgetpu-custom-op...) instead of LiteRT's DISPATCH_OP |
not usable here: LiteRT cannot recompile it or run it on the Google Tensor TPU, and the CPU cannot run it (so it is no --cpu-baseline or --fallback either; the scripts refuse it). Ask for the original model it was compiled from and compile that (step 2) |
The LiteRtStamp is authoritative (the _Google_Tensor_G5 suffix is only a
naming convention) and records the SoC but not the compiler flags: those
come from the <model>.compile.json sidecar that compile_model.py writes
next to its output; without it, say "flags unknown" rather than guessing.
Step 2 - Compile (Linux x86_64 or Google Colab via --colab)
2a. One-time environment
The SDK pip package contains no compiler by itself: at install time it unpacks
the Beta tarball that GOOGLE_TENSOR_SDK_BETA points to (same flow as the
public codelab). Ask the user for the tarball path first, then print the
install commands for the LiteRT version the skill pins in versions.json:
python3 SKILL_DIR/scripts/compile_model.py --print-install [--colab]
Local Linux x86_64 (--print-install): runs with any python3 and prints
comment lines (the pinned version, when and on what it was last verified, the
supported Python range, and whether this host and Python qualify) followed by:
python3 -m venv ~/.venvs/litert-tensor && source ~/.venvs/litert-tensor/bin/activate
pip install --upgrade pip
export GOOGLE_TENSOR_SDK_BETA=/abs/path/litert_plugin_compiler.tar.gz # from the Beta e-mail
pip install --no-cache-dir ai-edge-litert==<pinned> ai-edge-litert-sdk-google-tensor==<pinned>
Remote Google Colab (--print-install --colab) - for macOS, Windows,
non-x86 Linux, or when compiling on Colab without a local Linux venv: installs
google-colab-cli into ~/.venvs/colab-cli
(how --colab works).
Never ask the user to run a terminal command for OAuth: when --colab needs
one-time OAuth, show its printed URL in chat, ask the user for the code, and
re-run with --colab-auth-code <CODE>.
- Tell the user before installing anything. Run the printed commands with the
real tarball path and never type a LiteRT version yourself: the runtime and
SDK packages must be one version (
WARNING when they differ; NOTE when
they match each other but not the pin). Use -nightly packages only when
the user explicitly asks for nightlies.
- If
--print-install warns about the Python version, create the venv with
a python3.N in the supported range. Install errors:
references/troubleshooting.md.
If the user has no Beta access yet: say so plainly - the compiler cannot be
installed until Google e-mails the tarball link (sign-up form above) - and do
not install anything or allocate a Colab VM meanwhile (--colab checks for the
tarball before colab new). Still useful now: run check_compiled.py on their
files, confirm the host and phone SoC, and run --dry-run / --print-install.
2b. Compile
# Local Linux x86_64 (using the virtualenv's interpreter):
python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite \
--target Tensor_G5 --output-dir /abs/path/compiled
# Remote Google Colab VM (any OS, any local python3 + google-colab-cli):
python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite \
--target Tensor_G5 --colab --sdk-tarball /abs/path/litert_plugin_compiler.tar.gz \
--output-dir /abs/path/compiled
# -> /abs/path/compiled/model_Google_Tensor_G5.tflite
The script prints the compilation report, re-verifies the Tensor stamp, writes
<output>.compile.json (flags, ops offloaded / total, package versions,
SHA-256s) and never overwrites an existing output (exit 2) unless
--overwrite is given. Ops left on the CPU in the report mean the strict NPU
benchmark will fail (step 3, --allow-cpu-fallback). Every compiler, script
and Colab option: references/compile-flags.md.
If the compiler crashes with an internal error - or the user reports that it
did with the defaults (no need to reproduce the crash) - use
--truncation-type half --sharding-intensity maximum (public FAQ guidance)
and tell the user which flags produced the artifact. half changes the
numerics (16-bit floats on the TPU), so tell the user to validate the
model's outputs against the original before relying on it.
Report a compile-only job from the script's output and sidecar only:
**Compiled:** <output path> (<bytes from "Compiled model written to" line>)
**Target / flags:** <cli_flags from the sidecar, or "--target X (compiler defaults)">
**Ops offloaded:** <N / M from the "Ops offloaded to the TPU" line>
**LiteRT:** <"packages" from the sidecar, plus the script's WARNING/NOTE line if it printed one>
**Verified:** check_compiled.py -> <its Status line>
Exit codes: 0 ok, 2 bad input, 3 already compiled, 4 unsupported host,
5 LiteRT packages or Beta compiler library missing (message includes the
install fix), 6 compile failed (compiler log errors are printed),
7 output verification failed.
Step 3 - Benchmark on the device
python3 SKILL_DIR/scripts/benchmark_on_device.py \
--model /abs/path/compiled/model_Google_Tensor_G5.tflite \
[--cpu-baseline /abs/path/model.tflite] [--serial SERIAL] \
[--dispatch-lib /abs/path/libLiteRtDispatch_GoogleTensor.so] \
[--output-dir /abs/path/results]
The script refuses a model compiled for another SoC than the phone's (exit 4;
--dry-run never contacts the phone, so it proves nothing about the match),
fetches benchmark_model and libLiteRtDispatch_GoogleTensor.so once from the
bucket release in versions.json (cached; downloaded from <url> means a
first download - tell the user what was fetched), runs the strict NPU
benchmark, a second hooked NPU pass for TPU tile time and TPU energy, and the
optional CPU baseline on the uncompiled model in a fresh
/data/local/tmp/litert_benchmark/run-* directory, which it deletes even on
failure. It writes report.md, results.json and the logs to --output-dir
(nothing after exit 2/3/4 or --dry-run). Each step and flag:
references/benchmark-flags.md.
The script is NPU-only by design (--extra-arg cannot switch the accelerator,
and an uncompiled model is rejected with exit 2); GPU or non-Tensor runs
(benchmark_model --use_gpu=true etc. directly) are outside this skill. Other
options (--num-runs, --performance-mode, --signature,
--allow-cpu-fallback, --no-tile-time, --no-power, ...) are in --help.
- Offline host: pass both
--benchmark-bin and --dispatch-lib; a
--dry-run then shows user-provided: <path> for both binaries without
touching the phone or the network - the proof the user wants first.
- Binary versions: keep
benchmark_model and the dispatch library from
the same bucket release (the default does; --benchmark-url .../binaries/<release>/android_arm64/benchmark_model pins another one and
the dispatch URL follows). If a provenance md5 differs from
verified.binaries in versions.json, rerun with --refresh-binary
(details).
Step 4 - Report
Nothing in the report is typed by hand. benchmark_on_device.py prints a
Markdown header + table (also saved as report.md and as the markdown field
of results.json) in which every value is read from the device or the run.
Paste that block verbatim and add only the context the script cannot know:
**Summary:** <NPU "Inference avg" from the table> ms on the TPU vs <CPU "Inference avg"> ms on the CPU (<the N.NNx value from the script's "NPU speed-up vs CPU" line>); peak memory <NPU> vs <CPU> MB; <strict NPU | CPU fallback allowed>. TPU tile time <"TPU tile time / inference" row> ms per inference. TPU energy <"TPU energy / inference" row> mJ per inference.
<header and table exactly as printed by benchmark_on_device.py, including any "Unexpected log lines" section>
**Notes:** <what the Log check line reports: "none" when both logs show 0 unexpected lines and no profiler retry; otherwise each unexpected line with its meaning from references/benchmark-flags.md or troubleshooting.md, plus anything the user should validate>
When nothing ran (check exit 2, benchmark exit 2/3/4/5, no device), report
the same way - from output, not from memory:
**Not run:** <the script's ERROR line verbatim>
- Device: <the "Device: ..." line printed by benchmark_on_device.py, or the `adb devices -l` line>
- Model: <the "Status: ..." line from check_compiled.py>
- Nothing was pushed to or removed from the phone.
**Options:** <the next actions from step 1 / troubleshooting, most specific first>
- NEVER type, round, or "tidy up" numbers, and NEVER reuse values from an
earlier run or from any example in this skill - copy the script output.
Prose numbers (the Summary line) must be character-for-character the
table's values. Never edit
report.md (script output); if the user wants
the answer saved, write summary.md next to it.
- Before writing the Summary and Notes, read
Writing the report
(no CPU baseline, combined compile + benchmark jobs, run counts, CPU
fallback, mixed LiteRT releases, speed-up below 1x, TPU tile time/energy).
Step 5 - Package for an app (Gradle)
Two cases (Step 0, "New integration or update"); what the package contains and
how the script checks it: references/app-integration.md.
- 5a - New integration: an AI pack written by LiteRT's own exporter from
the files as they are (byte-identical, nothing recompiled), the Google
Tensor NPU runtime as an install-time feature module (from Google Maven),
the original model for the app's CPU path, a Kotlin loader, and an
INTEGRATION.md with every remaining step for the user's project.
- 5b - Update (the app already ships Google Tensor builds):
--update-project only reads the project and prepares the files to
replace, under the app's own paths and names, plus an UPDATE.md.
# 5a -> <fallback dir>/app_package/<pack name>/INTEGRATION.md and the modules
<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py \
--compiled /abs/path/compiled/model_Google_Tensor_G5.tflite \
[--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite] \
--fallback /abs/path/model.tflite \
[--delivery install-time|fast-follow|on-demand] [--pack-name NAME] \
[--model-name NAME] [--app-module app] [--kotlin-package com.example.litert] \
[--quick-test] [--output-dir /abs/path/app_package/NAME]
# 5b -> <fallback dir>/app_update/<pack>/UPDATE.md, update.json, files/<paths>
<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py \
--update-project /abs/path/to/app-project \
--compiled /abs/path/compiled/model_Google_Tensor_G5.tflite \
[--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite] \
--fallback /abs/path/model.tflite \
[--pack-name NAME] [--model-name NAME] [--cpu-asset PATH|none]
- Run
check_compiled.py on every file first (step 1), and package builds
that passed the strict benchmark (step 3).
- Run with
--dry-run first (any python3), then for real with a Python
that has ai-edge-litert. The script refuses mismatched inputs and never
replaces a directory it did not write.
- For 5b, show the user the verdict and the file list:
drop-in (only
model files change), structural (UPDATE.md section 2 lists the other
changes; list them and ask how to proceed before anything else) or
up-to-date. A Runtime check of device means the new builds must pass
the strict benchmark with the app's own runtime library first (UPDATE.md
section 4).
- Paste the script's
**Packaged:** ... or **Update plan:** ... block
verbatim, add one line (the delivery mode or the verdict in plain words,
and "next: <output-dir>/INTEGRATION.md" or UPDATE.md), then ask: "The
integration plan is ready (INTEGRATION.md). Do you want me to apply it
to your project?" (or "The update plan is ready (UPDATE.md). ..."). If
yes, go to step 6. Publishing to Google Play stays with the user.
Exit codes: 0 ok, 2 bad input (missing file, duplicate SoC, invalid name,
output exists, a build that differs from its sidecar, or a project
--update-project cannot update as is), 3 wrong model kind (uncompiled
--compiled, compiled --fallback), 5 LiteRT not importable or its exporter
API changed (the message has the fix), 7 post-export verification failed
(nothing is left at --output-dir).
Step 6 - Apply the plan to the user's project (when they say yes)
You apply the plan; the user approves it once. Before changing anything,
show the whole plan and get one approval. Then ask separately for
each risky item (the check marks them): upgrading the Android Gradle plugin
or Kotlin, adding Kotlin, raising minSdk, limiting the app to arm64-v8a
(devices without it can no longer install the app), changing the app's
LiteRT version, replacing project files that differ from the package,
removing a SoC. A declined item is skipped; say what will not work without
it.
6a - New integration: the plan is the fail items of a read-only check
(any python3, no LiteRT), each with its fix:
python3 SKILL_DIR/scripts/package_ai_pack.py --check-project /abs/path/to/app-project \
--package <output-dir of step 5a> [--app-module app] [--json]
6b - Update: the plan is UPDATE.md (the file table and section 2).
Follow the procedure in
references/app-integration.md
(backups, edits merged in the project's own style, re-check, one build). Never
delete or reformat unrelated code, and never touch signing configs.
Step 7 - Install and check on a Pixel (when the user agrees)
Builds the user's app, installs it on the attached Pixel the way Google Play
would (bundletool local testing, for the phone's device group), and reads
the loader's LITERT_TENSOR_RESULT line:
python3 SKILL_DIR/scripts/install_and_verify.py --project /abs/path/to/app-project \
[--app-module app] [--variant debug|release|<flavor>Debug] [--serial <id>] \
[--uninstall-first] [--dry-run]
Run --dry-run first: it shows the phone, its device group and whether the
NPU path is expected. Exit 7 means the app is already installed and local
testing cannot update it: ask the user (uninstalling deletes the app's data on
the phone), then rerun with --uninstall-first. The app must call
TensorModelLoader.load(); if it loads the model on a particular screen, tell
the user to open it while the script waits. Report the script's block
verbatim: "path":"npu" is the pass; for "path":"cpu" on a phone the app
has a build for, the npu_error says why. Requirements and exit codes:
references/app-integration.md.
Rules
- ALWAYS run
check_compiled.py before compiling or benchmarking, on every
model file involved - the compiler cannot recompile a compiled model, and
the TPU cannot run an uncompiled one.
- ALWAYS verify the device SoC matches the model's stamp (the benchmark
script does) - a
Tensor_G5 model on a Tensor_G6 phone fails with an
opaque dispatch error, wasting a debugging round.
- ALWAYS use the original uncompiled model for
--cpu-baseline - compiled
models contain DISPATCH_OP nodes that only the NPU can execute.
- ALWAYS clean up the per-run device directory (automatic; after an
interrupted run list the base first, then remove only its
run-* entries:
adb shell ls -la <base> and adb shell rm -rf <base>/run-*, where
<base> is /data/local/tmp/litert_benchmark or the --device-dir that
was used). NEVER rm -rf the base directory itself - users keep their own
binaries and models there.
- NEVER combine
--use_cpu=true with --require_full_delegation=true -
benchmark_model aborts on that combination.
- NEVER install packages into the system interpreter or without telling the
user; use a virtualenv so pinned LiteRT versions do not break other work.
- NEVER type LiteRT version numbers into install commands or download URLs
yourself - take them from
compile_model.py --print-install and the
scripts' default URLs, which all read versions.json, so every job uses the
pair that was verified together. Use another release only when the user
asks for one (--benchmark-url, or the pip pair at that version), and say
so in the report. Changing the pin is a maintainer task
(references/contributing.md).
- NEVER treat macOS, Windows, or ARM hosts as a compile blocker or give manual
browser Colab notebook steps - run
compile_model.py --colab directly.
- NEVER leave adb, benchmark, or Colab VM sessions running in the background -
every command here is bounded by a timeout and returns before the next step
(
compile_model.py --colab stops any Colab VM it creates unless
--keep-colab-session is passed).
- NEVER guess compile flags or overwrite an existing output without asking
(
compile_model.py exits 2; use --output-name or ask before
--overwrite).
- ALWAYS package for an app with
package_ai_pack.py (it calls LiteRT's
exporter and verifies the result), and with --update-project (step 5b)
when the app already ships Google Tensor builds; NEVER create or edit AI
pack folders, the device-targeting XML or the runtime modules by hand, and
NEVER copy a new package's ai_packs/ over an existing pack - the app's
group names and the place of its CPU copy can differ.
- ALWAYS ask before changing the user's project, show the whole plan, and
get one approval; ask separately for each risky item (step 6) and before
install_and_verify.py --uninstall-first. NEVER change signing configs.
- NEVER put a compiled build where the app runs on the CPU (base assets,
--fallback): only the original uncompiled model.
- NEVER publish to Google Play, sign, or upload anything for the user; the
skill stops at the package and the local-testing steps.
Troubleshooting
Every script error names its fix; look up the error line or exit code in
references/troubleshooting.md.
Public references
LiteRT NPU guide,
Google Tensor with LiteRT,
compilation flags,
Tensor SDK FAQ,
release notes / compatibility,
benchmark_model,
codelab,
Play for On-device AI.
Contributions
To contribute or modify this skill or its references, follow the
contribution guidelines before making changes.
1---2name: litert-google-tensor3description: >-4 Compiles LiteRT (.tflite) models ahead-of-time for Google Tensor TPUs5 (Tensor G5 on Pixel 10, Tensor G6 on Pixel 11) locally on Linux x86_64 or6 on Google Colab (--colab on macOS, Windows, or ARM hosts),7 benchmarks them on a USB-attached Pixel with LiteRT's benchmark_model8 (NPU latency, peak memory, TPU tile time, TPU energy, optional CPU baseline),9 and packages compiled models for an Android Gradle app as a Google Play AI10 pack with the Tensor NPU runtime module and a Kotlin loader, applies it to11 the app and verifies it on a Pixel. Use when the12 user asks to compile, AOT-compile, or prepare a .tflite model for the Pixel13 TPU/NPU, check whether a .tflite is already Tensor-compiled, measure14 on-device NPU performance of a compiled model, or package or deploy a15 compiled model into their Android app (or replace the one it ships). Don't16 use for Qualcomm, MediaTek, or Samsung NPUs, GPU-only benchmarking,17 LiteRT-LM or LLM (.litertlm) models, on-device JIT compilation, non-Gradle18 builds, or publishing to Google Play.19metadata:20 source: https://skillmd.com/skills/google/litert-google-tensor21 version: "1.0.0"22 category: AiAndMachineLearning23---2425# LiteRT on Google Tensor: compile, benchmark, package for apps2627Three scriptable workflows for Pixel TPUs:28291. **Compile** a plain `.tflite` into a Google Tensor AOT-compiled `.tflite`30 with the public LiteRT Python API (`ai_edge_litert.aot`), locally on Linux31 `x86_64` or on a Google Colab VM (`--colab`, via `google-colab-cli`) when32 the user is on macOS/Windows or lacks a Linux box.332. **Benchmark** a compiled model on a connected Pixel with LiteRT's34 `benchmark_model`, NPU-only, optionally next to a CPU baseline, including35 TPU tile time (pure TPU compute time) and TPU energy per inference.363. **Package** compiled builds for an Android (Gradle) app as a Google Play AI37 pack with the Google Tensor NPU runtime and a Kotlin loader (or, for an app38 that already ships Google Tensor builds, only the files to replace), apply39 the plan to the project when the user agrees, and check the app on a Pixel.4041Everything runs synchronously in the user's terminal; the scripts never start42background processes and always clean up what they pushed to the phone or43allocated on Colab. All paths below are relative to this skill's directory44(`SKILL_DIR`).4546Path | Purpose47---- | -------48[assets/versions.json](assets/versions.json) | LiteRT versions the scripts use (single source)49[scripts/check_compiled.py](scripts/check_compiled.py) | Is this `.tflite` already Tensor-compiled?50[scripts/compile_model.py](scripts/compile_model.py) | AOT compile (local Linux `x86_64` or `--colab`)51[scripts/benchmark_on_device.py](scripts/benchmark_on_device.py) | adb push -> benchmark -> parse -> cleanup52[scripts/package_ai_pack.py](scripts/package_ai_pack.py) | Compiled builds -> AI pack + Gradle modules for an app, or an update of one53[scripts/install_and_verify.py](scripts/install_and_verify.py) | Build the app, install it like Google Play, read the NPU/CPU result54[assets/templates/](assets/templates/) | Gradle, Kotlin, `INTEGRATION.md` and `UPDATE.md` templates (`package_ai_pack.py`)55[references/compile-flags.md](references/compile-flags.md) | API, compiler flags, script options, `--colab`56[references/benchmark-flags.md](references/benchmark-flags.md) | What the benchmark runs, binaries, writing the report, example output57[references/app-integration.md](references/app-integration.md) | Packaging for an app: contents, checks, delivery modes, applying the plan, checking on a Pixel58[references/troubleshooting.md](references/troubleshooting.md) | Symptom -> fix tables59[references/contributing.md](references/contributing.md) | Maintainers: adopting a new LiteRT release6061Every script supports `--help` and `--dry-run` and is Python 3 standard library62only, so it runs with any `python3`, with two exceptions that import LiteRT: a63local compile (`compile_model.py` without `--colab`, `--dry-run` or64`--print-install`) must use the virtualenv's interpreter65(`<venv>/bin/python SKILL_DIR/scripts/compile_model.py ...`, or activate the66venv first), and a real `package_ai_pack.py` run (not `--dry-run` or67`--check-project`) needs a Python with `ai-edge-litert` (the compile venv68works; the Beta SDK is not needed).6970## Prerequisites (tell the user up front)7172| Need | Compile (local) | Compile (`--colab`) | Benchmark | Package (app) |73|---|---|---|---|---|74| **Google Tensor SDK Beta access** - sign up at https://services.google.com/fb/forms/tensor_ml_sdk_experimental_access (overview: https://developers.google.com/edge/litert/next/tensor-sdk). Google emails a download link for `litert_plugin_compiler.tar.gz`; the compiler cannot be installed without it. | required | required (`--sdk-tarball` or `--sdk-url`) | - | - |75| Host OS | Linux **x86_64**, >= 16 GB RAM | any OS (`colab` CLI on PATH; runs on a remote Colab Linux `x86_64` VM) | any OS with `adb` | any OS where `ai-edge-litert` installs (the compile venv's Linux `x86_64` host works) |76| Python environment | Local venv with `ai-edge-litert` + `ai-edge-litert-sdk-google-tensor` at the version in `versions.json` (`compile_model.py --print-install`) | Any local `python3` + `google-colab-cli` (`compile_model.py --print-install --colab`); SDK is installed on the Colab VM automatically | any `python3` (stdlib only) | a Python with `ai-edge-litert` (the compile venv works; the SDK package is not needed) |77| `adb` on PATH, Pixel 10/11 with USB debugging enabled | - | - | required | step 7 (install and check the app) |78| A JDK the app's Gradle supports (`JAVA_HOME`), the Android SDK, the project's `./gradlew` | - | - | - | steps 6-7 (build, install); bundletool is downloaded once |79| `libLiteRtDispatch_GoogleTensor.so` - downloaded automatically from the public LiteRT binaries bucket (same directory as `benchmark_model`), or supplied with `--dispatch-lib` | - | - | required | - (the app gets it from Google Maven) |80| Internet | once for `pip install` | required (Colab VM + `pip install`) | once to fetch `benchmark_model` + dispatch lib (cached in `~/.cache/litert-google-tensor/`) unless supplied locally | - (the app's Gradle build downloads the LiteRT Maven artifacts) |8182Supported targets: `Tensor_G5` (Pixel 10 family) and `Tensor_G6` (Pixel 1183family), as listed by the public Tensor SDK docs; Pixel 9 and older are not.8485## Step 0 - Collect parameters8687Ask the user only for what is missing; never invent values:8889Parameter | How to get it90----------------------------------- | -------------91Model path | Absolute path to the `.tflite`. Confirm it exists.92Goal | check only, compile only, benchmark only, compile + benchmark, or package for an app (step 5). Infer it from the request ("how fast is it on the TPU vs CPU" = compile if needed + benchmark with a CPU baseline; "put it in my app" / "deploy it" = package); do not add an unrequested device run.93Target SoC | For a **compile**, the user's stated device wins (Pixel 10 family = `Tensor_G5`, Pixel 11 family = `Tensor_G6`), even if a different Pixel is attached. For a **benchmark**, the attached phone decides: `adb shell getprop ro.soc.model` -> `Tensor G5` => `Tensor_G5`, `Tensor G6` => `Tensor_G6` (the benchmark script reads and enforces this itself).94Output path | Default `<output-dir>/<stem>_Google_<target>.tflite`. `compile_model.py` refuses to overwrite an existing file (exit 2): run `check_compiled.py` on the existing file and reuse it, or pass `--output-name`, or `--overwrite` after asking. When a second build is needed next to an existing one, name it `<stem>_Google_<target>.<YYYYMMDD-HHMM>.tflite` (e.g. `--output-name model_Google_Tensor_G5.20260107-0930.tflite`) or, for a flag variant, `<stem>_Google_<target>.<flag-summary>.tflite` (`...G5.half-maxsharding.tflite`); keep the `_Google_<target>` part so the file stays recognisable.95Device serial | Only if `adb devices` lists more than one device.96Dispatch library / benchmark binary | Optional. Both are downloaded automatically from the same public bucket directory and cached. Only ask if the user wants a specific build, or if the host is offline: then `--benchmark-bin <benchmark_model>` plus `--dispatch-lib <libLiteRtDispatch_GoogleTensor.so \| dir \| .zip \| .aar>` (also env `LITERT_GOOGLE_TENSOR_DISPATCH_LIB`) use local copies and no network is touched.97CPU baseline | Optional; needs the path of the **original uncompiled** model (ask for it if only a compiled file was given; the benchmark script rejects a compiled file here with exit 2).98Run counts / mode | Defaults 5 warmup / 50 measured, `burst` performance mode. `benchmark_model` treats counts as **minimums** and also runs at least 0.5 s warmup / 1 s measured, so fast models execute more runs than requested. If the user states exact counts, say so and ask whether to disable the floors (`--extra-arg=--min_secs=0 --extra-arg=--warmup_min_secs=0`); "steady-state" / "sustained" requests map to `--performance-mode sustained_performance`.99TPU tile time | On by default, nothing to ask: a second, hooked NPU pass. `--no-tile-time` turns it off; the hooked pass itself (it repeats the NPU run, so slow models take about twice as long) is skipped only with `--no-tile-time --no-power`.100TPU energy | On by default, nothing to ask: the power metric of the same hooked pass (the TPU rail's energy per inference). `--no-power` skips it. The script leaves it off, and says why, for a dispatch library whose power hook predates the fix.101Compile mode | Local Linux `x86_64` by default. Use `--colab` when the host is not Linux `x86_64` (macOS/Windows/ARM) or when the user asks to compile on Google Colab; ask for the Beta tarball path (`--sdk-tarball` / `GOOGLE_TENSOR_SDK_BETA`) or URL (`--sdk-url` / `GOOGLE_TENSOR_ML_SDK_URL`) before running.102Compiler flags | Default to the compiler defaults; only set flags the user asks for or that troubleshooting requires (see [references/compile-flags.md](references/compile-flags.md)).103App deployment level | Packaging jobs only; nothing to ask up front. After step 5, tell the user the plan is ready (`INTEGRATION.md`, or `UPDATE.md` for an update) and ask whether you should apply it to their project (step 6). After that, offer to install the app on their Pixel and check it uses the NPU (step 7).104New integration or update | Packaging jobs only. Ask whether the app already ships Google Tensor builds (an AI pack with folders like `model#group_Google_Tensor_G5` under its `src/main/assets`). If it does, it is an **update** (step 5b): ask for the project root (the folder with `settings.gradle.kts` or `settings.gradle`). The pack, model file, device groups, delivery mode and app module then come from the project, so do not ask for them.105Builds to package | One compiled build per target SoC (`Tensor_G5`, `Tensor_G6`) plus the **original uncompiled** model they were compiled from (the app's CPU path). Compile missing targets first (step 2); a build is per SoC, so a Pixel 11 needs a `Tensor_G6` build. For an update, build every SoC the app already ships.106Delivery mode | New integrations only (an update keeps the app's). When Google Play puts the model on the phone: `install-time` (default; installed with the app, ready at first launch), `fast-follow` (downloaded right after install) or `on-demand` (downloaded when the app asks). Explain each in one sentence ([references/app-integration.md](references/app-integration.md#delivery-modes-asked-in-step-0)) and use the default when the user has no preference.107Pack / model names, app module | New integrations only. Defaults: pack name = the original's file name made valid, model name = pack name, app module `app`, Kotlin package `com.example.litert`. Ask only when the user's project differs (another app module name, their package).108Quick test (optional) | New integrations only. Offer it when the user wants to try the model on their own Pixel before setting up bundletool: `--quick-test` adds a copy of each build and the steps for `./gradlew installDebug -PtensorQuickTest=<SoC>`, which puts the build and the NPU runtime in the debug APK. Local testing only; release builds are unaffected.109110## Commands or a plan only (the user says not to run anything)111112Run nothing - not even the read-only scripts - and give the same steps as a113real run, in the same order, as commands the user can paste:1141151. Start with `check_compiled.py` on every input model (step 1), **before**116 any install or compile command; checking only the compiled output117 afterwards is not enough.1182. For installs, give `python3 SKILL_DIR/scripts/compile_model.py119 --print-install [--colab]` as the command that produces them (step 2a),120 then the lines it prints with the user's paths filled in (venv,121 `export GOOGLE_TENSOR_SDK_BETA=<their tarball>` before `pip install`) and122 each version written as `<version printed by --print-install>`. Never123 copy a version number from `versions.json` or from memory.1243. Give no results, not even estimates: latency, speed-up, memory and ops125 offloaded come only from the scripts' output. Say where they will appear126 (for a benchmark, the table `benchmark_on_device.py` prints and saves as127 `report.md`).1284. When the plan changes the user's app project (step 6), say that nothing129 in the project changes until they approve, and end by asking whether you130 should apply it.131132## Step 1 - Check whether the model is already compiled133134```bash135python3 SKILL_DIR/scripts/check_compiled.py /abs/path/model.tflite [--expect-target Tensor_G5]136```137138Run it on every model file involved (the model to compile or benchmark, the139CPU-baseline file, and an existing output file you are about to reuse). For a140benchmark job read the phone first (`adb shell getprop ro.soc.model` prints141`Tensor G5` or `Tensor G6`) and pass the underscore form as `--expect-target`142(`Tensor G5` -> `--expect-target Tensor_G5`), so one check answers both143"compiled?" and "for this phone?" (a plain model exits 1 either way); the144benchmark script enforces the match again anyway (exit 4).145146| Exit | Meaning | Next action |147|---|---|---|148| 0 | compiled for Google Tensor (SoC printed) | skip compile; report the SoC (and the `Compile info` line if a `.compile.json` sidecar exists). Continue to step 3 only if the goal includes a benchmark. No sidecar (compiled elsewhere, flags unknown): for a plain "how fast is it" job, benchmark the file as-is and say "compile flags unknown"; when the user needs to know how it was built (or wants a reproducible comparison), compile the original with the defaults under a new `--output-name` and compare SHA-256 (`sha256sum` of both files, or `output_sha256` in the new sidecar): compilation is deterministic, so an identical hash proves the existing file is a default build ([references/compile-flags.md](references/compile-flags.md)). Never `--overwrite` to find out. |149| 1 | plain (uncompiled) model | go to step 2 if compiling; this is the correct file for `--cpu-baseline` |150| 2 | compiled for a different SoC than `--expect-target` | compiled models are SoC-specific (a `Tensor_G5` file never runs on a `Tensor_G6` TPU, and a `Tensor_G6` file never runs on a `Tensor_G5` TPU; there is no runtime fallback, and `--allow-mismatch` only produces dispatch errors). Tell the user which SoC the phone has, then offer, in this order: an existing `*_Google_<phone target>.tflite` build of the same model in the project (verify it with this check first; do not benchmark it unless the user says so), attaching the matching phone, or recompiling the **original** uncompiled export for the phone's target. |151| 3 | not a `.tflite` / missing | fix the path |152| 4 | compiled by another toolchain: an Edge TPU custom op (`edgetpu-custom-op...`) instead of LiteRT's `DISPATCH_OP` | not usable here: LiteRT cannot recompile it or run it on the Google Tensor TPU, and the CPU cannot run it (so it is no `--cpu-baseline` or `--fallback` either; the scripts refuse it). Ask for the original model it was compiled from and compile that (step 2) |153154The `LiteRtStamp` is authoritative (the `_Google_Tensor_G5` suffix is only a155naming convention) and records the SoC but **not** the compiler flags: those156come from the `<model>.compile.json` sidecar that `compile_model.py` writes157next to its output; without it, say "flags unknown" rather than guessing.158159## Step 2 - Compile (Linux x86_64 or Google Colab via `--colab`)160161### 2a. One-time environment162163The SDK pip package contains no compiler by itself: at install time it unpacks164the Beta tarball that `GOOGLE_TENSOR_SDK_BETA` points to (same flow as the165public codelab). Ask the user for the tarball path first, then print the166install commands for the LiteRT version the skill pins in `versions.json`:167168```bash169python3 SKILL_DIR/scripts/compile_model.py --print-install [--colab]170```171172**Local Linux `x86_64`** (`--print-install`): runs with any `python3` and prints173comment lines (the pinned version, when and on what it was last verified, the174supported Python range, and whether this host and Python qualify) followed by:175176```text177python3 -m venv ~/.venvs/litert-tensor && source ~/.venvs/litert-tensor/bin/activate178pip install --upgrade pip179export GOOGLE_TENSOR_SDK_BETA=/abs/path/litert_plugin_compiler.tar.gz # from the Beta e-mail180pip install --no-cache-dir ai-edge-litert==<pinned> ai-edge-litert-sdk-google-tensor==<pinned>181```182183**Remote Google Colab** (`--print-install --colab`) - for macOS, Windows,184non-x86 Linux, or when compiling on Colab without a local Linux venv: installs185`google-colab-cli` into `~/.venvs/colab-cli`186([how `--colab` works](references/compile-flags.md#compiling-on-google-colab---colab)).187Never ask the user to run a terminal command for OAuth: when `--colab` needs188one-time OAuth, show its printed URL in chat, ask the user for the code, and189re-run with `--colab-auth-code <CODE>`.190191* Tell the user before installing anything. Run the printed commands with the192 real tarball path and never type a LiteRT version yourself: the runtime and193 SDK packages must be one version (`WARNING` when they differ; `NOTE` when194 they match each other but not the pin). Use `-nightly` packages only when195 the user explicitly asks for nightlies.196* If `--print-install` warns about the Python version, create the venv with197 a `python3.N` in the supported range. Install errors:198 [references/troubleshooting.md](references/troubleshooting.md#environment-and-install).199200**If the user has no Beta access yet:** say so plainly - the compiler cannot be201installed until Google e-mails the tarball link (sign-up form above) - and do202not install anything or allocate a Colab VM meanwhile (`--colab` checks for the203tarball before `colab new`). Still useful now: run `check_compiled.py` on their204files, confirm the host and phone SoC, and run `--dry-run` / `--print-install`.205206### 2b. Compile207208```bash209# Local Linux x86_64 (using the virtualenv's interpreter):210python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite \211 --target Tensor_G5 --output-dir /abs/path/compiled212213# Remote Google Colab VM (any OS, any local python3 + google-colab-cli):214python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite \215 --target Tensor_G5 --colab --sdk-tarball /abs/path/litert_plugin_compiler.tar.gz \216 --output-dir /abs/path/compiled217# -> /abs/path/compiled/model_Google_Tensor_G5.tflite218```219220The script prints the compilation report, re-verifies the Tensor stamp, writes221`<output>.compile.json` (flags, ops offloaded / total, package versions,222SHA-256s) and never overwrites an existing output (exit 2) unless223`--overwrite` is given. Ops left on the CPU in the report mean the strict NPU224benchmark will fail (step 3, `--allow-cpu-fallback`). Every compiler, script225and Colab option: [references/compile-flags.md](references/compile-flags.md).226227* If the compiler crashes with an internal error - or the user reports that it228 did with the defaults (no need to reproduce the crash) - use229 `--truncation-type half --sharding-intensity maximum` (public FAQ guidance)230 and tell the user which flags produced the artifact. `half` changes the231 numerics (16-bit floats on the TPU), so tell the user to validate the232 model's outputs against the original before relying on it.233* Report a compile-only job from the script's output and sidecar only:234235 ```markdown236 **Compiled:** <output path> (<bytes from "Compiled model written to" line>)237 **Target / flags:** <cli_flags from the sidecar, or "--target X (compiler defaults)">238 **Ops offloaded:** <N / M from the "Ops offloaded to the TPU" line>239 **LiteRT:** <"packages" from the sidecar, plus the script's WARNING/NOTE line if it printed one>240 **Verified:** check_compiled.py -> <its Status line>241 ```242243Exit codes: 0 ok, 2 bad input, 3 already compiled, 4 unsupported host,2445 LiteRT packages or Beta compiler library missing (message includes the245install fix), 6 compile failed (compiler log errors are printed),2467 output verification failed.247248## Step 3 - Benchmark on the device249250```bash251python3 SKILL_DIR/scripts/benchmark_on_device.py \252 --model /abs/path/compiled/model_Google_Tensor_G5.tflite \253 [--cpu-baseline /abs/path/model.tflite] [--serial SERIAL] \254 [--dispatch-lib /abs/path/libLiteRtDispatch_GoogleTensor.so] \255 [--output-dir /abs/path/results]256```257258The script refuses a model compiled for another SoC than the phone's (exit 4;259`--dry-run` never contacts the phone, so it proves nothing about the match),260fetches `benchmark_model` and `libLiteRtDispatch_GoogleTensor.so` once from the261bucket release in `versions.json` (cached; `downloaded from <url>` means a262first download - tell the user what was fetched), runs the strict NPU263benchmark, a second hooked NPU pass for TPU tile time and TPU energy, and the264optional CPU baseline on the **uncompiled** model in a fresh265`/data/local/tmp/litert_benchmark/run-*` directory, which it deletes even on266failure. It writes `report.md`, `results.json` and the logs to `--output-dir`267(nothing after exit 2/3/4 or `--dry-run`). Each step and flag:268[references/benchmark-flags.md](references/benchmark-flags.md#what-the-script-does).269270The script is NPU-only by design (`--extra-arg` cannot switch the accelerator,271and an uncompiled model is rejected with exit 2); GPU or non-Tensor runs272(`benchmark_model --use_gpu=true` etc. directly) are outside this skill. Other273options (`--num-runs`, `--performance-mode`, `--signature`,274`--allow-cpu-fallback`, `--no-tile-time`, `--no-power`, ...) are in `--help`.275276* **Offline host:** pass both `--benchmark-bin` and `--dispatch-lib`; a277 `--dry-run` then shows `user-provided: <path>` for both binaries without278 touching the phone or the network - the proof the user wants first.279* **Binary versions:** keep `benchmark_model` and the dispatch library from280 the **same** bucket release (the default does; `--benchmark-url281 .../binaries/<release>/android_arm64/benchmark_model` pins another one and282 the dispatch URL follows). If a provenance md5 differs from283 `verified.binaries` in `versions.json`, rerun with `--refresh-binary`284 ([details](references/benchmark-flags.md#choosing-binary-versions)).285286## Step 4 - Report287288Nothing in the report is typed by hand. `benchmark_on_device.py` prints a289Markdown header + table (also saved as `report.md` and as the `markdown` field290of `results.json`) in which every value is read from the device or the run.291Paste that block verbatim and add only the context the script cannot know:292293```markdown294**Summary:** <NPU "Inference avg" from the table> ms on the TPU vs <CPU "Inference avg"> ms on the CPU (<the N.NNx value from the script's "NPU speed-up vs CPU" line>); peak memory <NPU> vs <CPU> MB; <strict NPU | CPU fallback allowed>. TPU tile time <"TPU tile time / inference" row> ms per inference. TPU energy <"TPU energy / inference" row> mJ per inference.295296<header and table exactly as printed by benchmark_on_device.py, including any "Unexpected log lines" section>297298**Notes:** <what the Log check line reports: "none" when both logs show 0 unexpected lines and no profiler retry; otherwise each unexpected line with its meaning from references/benchmark-flags.md or troubleshooting.md, plus anything the user should validate>299```300301When nothing ran (check exit 2, benchmark exit 2/3/4/5, no device), report302the same way - from output, not from memory:303304```markdown305**Not run:** <the script's ERROR line verbatim>306307- Device: <the "Device: ..." line printed by benchmark_on_device.py, or the `adb devices -l` line>308- Model: <the "Status: ..." line from check_compiled.py>309- Nothing was pushed to or removed from the phone.310311**Options:** <the next actions from step 1 / troubleshooting, most specific first>312```313314* NEVER type, round, or "tidy up" numbers, and NEVER reuse values from an315 earlier run or from any example in this skill - copy the script output.316 Prose numbers (the Summary line) must be character-for-character the317 table's values. Never edit `report.md` (script output); if the user wants318 the answer saved, write `summary.md` next to it.319* Before writing the Summary and Notes, read320 [Writing the report](references/benchmark-flags.md#writing-the-report)321 (no CPU baseline, combined compile + benchmark jobs, run counts, CPU322 fallback, mixed LiteRT releases, speed-up below 1x, TPU tile time/energy).323324## Step 5 - Package for an app (Gradle)325326Two cases (Step 0, "New integration or update"); what the package contains and327how the script checks it: [references/app-integration.md](references/app-integration.md).328329* **5a - New integration:** an AI pack written by LiteRT's own exporter from330 the files as they are (byte-identical, nothing recompiled), the Google331 Tensor NPU runtime as an install-time feature module (from Google Maven),332 the original model for the app's CPU path, a Kotlin loader, and an333 `INTEGRATION.md` with every remaining step for the user's project.334* **5b - Update** (the app already ships Google Tensor builds):335 `--update-project` only reads the project and prepares the files to336 replace, under the app's own paths and names, plus an `UPDATE.md`.337338```bash339# 5a -> <fallback dir>/app_package/<pack name>/INTEGRATION.md and the modules340<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py \341 --compiled /abs/path/compiled/model_Google_Tensor_G5.tflite \342 [--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite] \343 --fallback /abs/path/model.tflite \344 [--delivery install-time|fast-follow|on-demand] [--pack-name NAME] \345 [--model-name NAME] [--app-module app] [--kotlin-package com.example.litert] \346 [--quick-test] [--output-dir /abs/path/app_package/NAME]347348# 5b -> <fallback dir>/app_update/<pack>/UPDATE.md, update.json, files/<paths>349<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py \350 --update-project /abs/path/to/app-project \351 --compiled /abs/path/compiled/model_Google_Tensor_G5.tflite \352 [--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite] \353 --fallback /abs/path/model.tflite \354 [--pack-name NAME] [--model-name NAME] [--cpu-asset PATH|none]355```3563571. Run `check_compiled.py` on every file first (step 1), and package builds358 that passed the strict benchmark (step 3).3592. Run with `--dry-run` first (any `python3`), then for real with a Python360 that has `ai-edge-litert`. The script refuses mismatched inputs and never361 replaces a directory it did not write.3623. For 5b, show the user the verdict and the file list: `drop-in` (only363 model files change), `structural` (`UPDATE.md` section 2 lists the other364 changes; list them and ask how to proceed before anything else) or365 `up-to-date`. A `Runtime` check of `device` means the new builds must pass366 the strict benchmark with the app's own runtime library first (`UPDATE.md`367 section 4).3684. Paste the script's `**Packaged:** ...` or `**Update plan:** ...` block369 verbatim, add one line (the delivery mode or the verdict in plain words,370 and "next: `<output-dir>/INTEGRATION.md`" or `UPDATE.md`), then ask: "The371 integration plan is ready (`INTEGRATION.md`). Do you want me to apply it372 to your project?" (or "The update plan is ready (`UPDATE.md`). ..."). If373 yes, go to step 6. Publishing to Google Play stays with the user.374375Exit codes: 0 ok, 2 bad input (missing file, duplicate SoC, invalid name,376output exists, a build that differs from its sidecar, or a project377`--update-project` cannot update as is), 3 wrong model kind (uncompiled378`--compiled`, compiled `--fallback`), 5 LiteRT not importable or its exporter379API changed (the message has the fix), 7 post-export verification failed380(nothing is left at `--output-dir`).381382## Step 6 - Apply the plan to the user's project (when they say yes)383384You apply the plan; the user approves it once. Before changing anything,385show the **whole plan** and get **one approval**. Then ask **separately** for386each risky item (the check marks them): upgrading the Android Gradle plugin387or Kotlin, adding Kotlin, raising `minSdk`, limiting the app to arm64-v8a388(devices without it can no longer install the app), changing the app's389LiteRT version, replacing project files that differ from the package,390removing a SoC. A declined item is skipped; say what will not work without391it.392393* **6a - New integration:** the plan is the `fail` items of a read-only check394 (any `python3`, no LiteRT), each with its fix:395396 ```bash397 python3 SKILL_DIR/scripts/package_ai_pack.py --check-project /abs/path/to/app-project \398 --package <output-dir of step 5a> [--app-module app] [--json]399 ```400401* **6b - Update:** the plan is `UPDATE.md` (the file table and section 2).402403Follow the procedure in404[references/app-integration.md](references/app-integration.md#applying-the-plan-skillmd-step-6)405(backups, edits merged in the project's own style, re-check, one build). Never406delete or reformat unrelated code, and never touch signing configs.407408## Step 7 - Install and check on a Pixel (when the user agrees)409410Builds the user's app, installs it on the attached Pixel the way Google Play411would (bundletool local testing, for the phone's device group), and reads412the loader's `LITERT_TENSOR_RESULT` line:413414```bash415python3 SKILL_DIR/scripts/install_and_verify.py --project /abs/path/to/app-project \416 [--app-module app] [--variant debug|release|<flavor>Debug] [--serial <id>] \417 [--uninstall-first] [--dry-run]418```419420Run `--dry-run` first: it shows the phone, its device group and whether the421NPU path is expected. Exit 7 means the app is already installed and local422testing cannot update it: ask the user (uninstalling deletes the app's data on423the phone), then rerun with `--uninstall-first`. The app must call424`TensorModelLoader.load()`; if it loads the model on a particular screen, tell425the user to open it while the script waits. Report the script's block426verbatim: `"path":"npu"` is the pass; for `"path":"cpu"` on a phone the app427has a build for, the `npu_error` says why. Requirements and exit codes:428[references/app-integration.md](references/app-integration.md#checking-on-a-pixel-skillmd-step-7).429430## Rules431432* ALWAYS run `check_compiled.py` before compiling or benchmarking, on every433 model file involved - the compiler cannot recompile a compiled model, and434 the TPU cannot run an uncompiled one.435* ALWAYS verify the device SoC matches the model's stamp (the benchmark436 script does) - a `Tensor_G5` model on a `Tensor_G6` phone fails with an437 opaque dispatch error, wasting a debugging round.438* ALWAYS use the original uncompiled model for `--cpu-baseline` - compiled439 models contain `DISPATCH_OP` nodes that only the NPU can execute.440* ALWAYS clean up the per-run device directory (automatic; after an441 interrupted run list the base first, then remove only its `run-*` entries:442 `adb shell ls -la <base>` and `adb shell rm -rf <base>/run-*`, where443 `<base>` is `/data/local/tmp/litert_benchmark` or the `--device-dir` that444 was used). NEVER `rm -rf` the base directory itself - users keep their own445 binaries and models there.446* NEVER combine `--use_cpu=true` with `--require_full_delegation=true` -447 `benchmark_model` aborts on that combination.448* NEVER install packages into the system interpreter or without telling the449 user; use a virtualenv so pinned LiteRT versions do not break other work.450* NEVER type LiteRT version numbers into install commands or download URLs451 yourself - take them from `compile_model.py --print-install` and the452 scripts' default URLs, which all read `versions.json`, so every job uses the453 pair that was verified together. Use another release only when the user454 asks for one (`--benchmark-url`, or the pip pair at that version), and say455 so in the report. Changing the pin is a maintainer task456 ([references/contributing.md](references/contributing.md)).457* NEVER treat macOS, Windows, or ARM hosts as a compile blocker or give manual458 browser Colab notebook steps - run `compile_model.py --colab` directly.459* NEVER leave adb, benchmark, or Colab VM sessions running in the background -460 every command here is bounded by a timeout and returns before the next step461 (`compile_model.py --colab` stops any Colab VM it creates unless462 `--keep-colab-session` is passed).463* NEVER guess compile flags or overwrite an existing output without asking464 (`compile_model.py` exits 2; use `--output-name` or ask before465 `--overwrite`).466* ALWAYS package for an app with `package_ai_pack.py` (it calls LiteRT's467 exporter and verifies the result), and with `--update-project` (step 5b)468 when the app already ships Google Tensor builds; NEVER create or edit AI469 pack folders, the device-targeting XML or the runtime modules by hand, and470 NEVER copy a new package's `ai_packs/` over an existing pack - the app's471 group names and the place of its CPU copy can differ.472* ALWAYS ask before changing the user's project, show the whole plan, and473 get one approval; ask separately for each risky item (step 6) and before474 `install_and_verify.py --uninstall-first`. NEVER change signing configs.475* NEVER put a compiled build where the app runs on the CPU (base assets,476 `--fallback`): only the original uncompiled model.477* NEVER publish to Google Play, sign, or upload anything for the user; the478 skill stops at the package and the local-testing steps.479480## Troubleshooting481482Every script error names its fix; look up the error line or exit code in483[references/troubleshooting.md](references/troubleshooting.md).484485## Public references486487[LiteRT NPU guide](https://developers.google.com/edge/litert/next/npu),488[Google Tensor with LiteRT](https://developers.google.com/edge/litert/next/tensor-sdk),489[compilation flags](https://developers.google.com/edge/tensor-sdk/compilation-flags),490[Tensor SDK FAQ](https://developers.google.com/edge/tensor-sdk/tensor-faqs),491[release notes / compatibility](https://developers.google.com/edge/tensor-sdk/release-notes),492[benchmark_model](https://developers.google.com/edge/litert/next/benchmark),493[codelab](https://codelabs.developers.google.com/codelabs/google-tensor-ml-sdk),494[Play for On-device AI](https://developer.android.com/google/play/on-device-ai).495496## Contributions497498To contribute or modify this skill or its references, follow the499[contribution guidelines](references/contributing.md) before making changes.