Agent Contribution Guide: LiteRT Quantizer Recipe Exploration
Welcome, AI Agent! This skill governs how you autonomously explore and generate
the optimal selective quantization recipe for any given LiteRT FlatBuffer
(.tflite) model using the LiteRT Quantizer framework. Strict adherence
to these protocols ensures code is high-quality, memory safe, and meets
verification protocols.
Your goal is to build an iterative search loop in Python that actively
interrogates the baseline model for numerical degradation, updates the recipe on
the fly to protect highly sensitive tensors, and produces a bounding-box of
Pareto-optimal models for the user.
Skill Index
Leverage these project-specific skill guides to navigate and execute tasks
effectively.
Core guides (read ALL of these before proceeding):
| File |
Contents |
README.md |
Overview of the LiteRT Quantizer framework and the repo's structure |
README.md |
What configurations are supported for each operator |
model_scale_tiers.md |
Scale tier classification (S/M/L) with per-tier search budgets, batched squashing, and memory guardrails |
algorithms.md |
Registry of available quantization algorithms and when to select each one |
error_metric_selection.md |
Registry of validation error metrics, task-specific selection, distribution-relative thresholding, and per-modality stopping metrics |
file_naming.md |
Standardized conventions for naming exported models, recipes, evaluation artifacts, and validation reports |
quantization_pattern.md |
Empirical patterns: what works, what doesn't, and why |
snr_best_practices.md |
How to compute, parse, and threshold per-tensor metrics to locate fragile layers |
kurtosis_screening.md |
Zero-validation, data-free layer sensitivity screening via weight kurtosis, plus the trimmed z-score outlier detection routine |
regex_targeting.md |
How to write readable, unified block-level regex patterns to avoid scattered precision overlaps |
size_estimation.md |
How to compute theoretical compressed model size |
experiment_runner.md |
Memory-safe experiment loop, batched squashing, the algorithm search axis |
pareto_curve_plotting.md |
Guidelines on formatting the Pareto visualization, drawing frontiers, and handling outliers |
output_report.md |
How to format and organize the output produced |
Pluggable task evaluation family (read ONLY the file matching the model's
modality):
| File |
Applies To |
image_segmentation_eval.md |
Segmentation / detection models (masks, mIoU/Dice, heatmaps) |
llm_eval.md |
Generative LLMs (multi-signature handling, token agreement, KL, sample generations) |
classification_eval.md |
Classifiers (Top-1/Top-5 agreement, confidence drift) |
Conditional guides (read when applicable):
| File |
Read When |
calibration_data.md |
Using static quantization (calibration dataset rules, leakage prevention, manifests) |
latency_benchmarking.md |
The user states a latency requirement or asks for on-device numbers |
Phase 0: Understand the Framework and Classify the Model
Before beginning, explicitly use specific file reading tools to read all core
guides in the Skill Index, plus the ONE task evaluation guide matching the
model's modality, plus any applicable conditional guides.
Then, before writing any plan:
- Classify the modality (segmentation, LLM, classification, embedding,
audio, regression) from the model's signatures, I/O tensor shapes, and the
user's description. This selects your metrics
(
error_metric_selection.md) and evaluation family.
- Classify the scale tier (S/M/L per
model_scale_tiers.md) from the
float model file size and signature count. This selects your search
budget, batching strategy, and protection-mechanism priority.
1. Implementation Plan Guidelines
Because complex selective quantization tasks can cause context loss during
execution, your first action MUST be to draft a highly explicit, self-contained
Implementation Plan. Since you will follow this plan closely, you must bake all
necessary context directly into it.
You MUST output this plan as an artifact file (e.g.
implementation_plan.md, using your file-writing tool). In interactive
sessions, HALT after writing the plan and explicitly ask the user for
permission to proceed before you make any code edits or run any loops. If
the user has already provided explicit bounds AND explicitly requested
unattended/autonomous execution (e.g. a CI pipeline), you may proceed without
halting, but the plan artifact is still mandatory. You must strictly follow
this plan once approved.
Your printed plan must contain:
Step 1: Context & Role Declaration
- Role Judgment: Declare your role (Contributor, Maintainer, or Reviewer).
- Model Classification: State the detected modality, the scale tier, and
the resulting metric selections and search budgets.
- Context Summary: Explicitly write out a 2-3 sentence summary of the
specific constraints you read from the skill docs. You MUST explicitly state
that your artifact names will obey the strict rules from
file_naming.md
and explicitly confirm you will produce a Pareto visualization per
pareto_curve_plotting.md.
Step 2: Technical Design & File List
- Target Files: Explicitly list all files you plan to create or modify.
Please adhere strictly to the
output_report.md requirements.
- Optimization Constraints: Briefly outline how you will handle target
file size bounds, acceptable error thresholds across MULTIPLE metrics, and
generic regex targeting to prevent data leakage.
Step 3: Experiment Harness Architecture
- Metric Selection: Infer from the float model what metrics you should use
for validation following the guidelines of
error_metric_selection.md,
including the modality-appropriate stopping metric.
- Experiment Blueprint: Explicitly outline your script structure focusing
on proper API sequential updates, memory-safe
Quantizer instantiation per
permutation, strictly sequential qt.validate() calls (one validation at a
time), explicit garbage collection (import gc; gc.collect()), and
selecting layer candidates. Most importantly, explicitly confirm you are
implementing the stringent Two-Phase loop mandated by experiment_runner.md
(Phase 1: Freezing protection set, Phase 2: Iterative Greedy INT4 squashing
sorted by robustness, batched per your scale tier). Do not hallucinate
trial-and-error hacks.
- Budget Declaration: State the per-tier validation cap and sample count
you will operate under (from
model_scale_tiers.md).
Step 4: Verification Plan
- Review Hand-off: Document your explicit commitment to run the Two-Pass
Review Loop (see "Agent Verification Protocol" below) on your execution
script before you present the final report. If your harness supports
launching subagents, commit to delegating the review to a reviewer
subagent; otherwise commit to performing the structured self-review pass
against the Reviewer checklist. ONLY start the review after you think
you're done with the quantization exploration process.
Step 5: Post-Execution Walkthrough
- Acknowledge that the final response will adhere to the Post-Execution
Walkthrough Guidelines (below).
Phase 1: Setup & Bounding Box
- Ask the User for Bounds: If the user did not specify, explicitly ask for
either their Minimum Model Compression Ratio / Model Size OR their Max
Tolerable Error Bound. Our objective is to find either the best quality
model given the minimum compression ratio or the smallest model given the
max tolerable error bound.
- Acquire Verification Preprocessing: In order to use
qt.validate(), you
need numpy tensor inputs matching the model's signature(s). You may copy
the input processing logic from existing manual human scripts in the
directory (for example processing.py), or construct tokenized prompts
for LLMs per llm_eval.md.
- CRITICAL CONSTRAINT (NO LEAKAGE): You MUST NOT copy the human's
manual selective recipe adjustments from existing sources or tutorials
to blindly inject layer names. You must discover which layers to
protect dynamically based on your own validation metric parsing!
This equally applies to the empirical anecdotes and example values
inside these skill guides: treat them as unverified priors that
motivate WHERE to look, never as pre-validated protection lists or
expected metric values. Every protection you commit must be
justified by a measurement from YOUR current run.
- Single-source-input warning: If the user provides only ONE test
input, deterministic augmentations of it are acceptable for the
search loop, but the resulting recipe may be mildly overfit to that
input. You MUST state this limitation explicitly in the final report
and recommend re-validating the chosen profiles on a small held-out
set before production use.
- Effort & Sizing Logic: The user's size bounds represent a limit, NOT a
target to match exactly. You are authorized to return models that are
smaller than requested! The primary goal is to maximize accuracy (or the
target metric) while guaranteeing the model is small enough. If you can make
it even smaller while retaining acceptable accuracy, do so. Never
artificially inflate the model just to hit a bound. Try multiple iterations
until you have found the best possible quantization recipe, within the
validation budget of your scale tier.
Phase 2: Building the Iteration Harness
Do not try to guess the recipe blindly or write concurrent sweeps. You must
construct a dedicated, repeatable Python evaluation script (e.g.
explore_litertq_model.py) that operates on ONE recipe at a time. Do not run
parallel configurations or concurrent validations.
- Load the float baseline
.tflite model and run data through preprocessing.
- Instantiate
quantizer.Quantizer() and load ONE default baseline
starting recipe via the API.
- Inject targeted
qt.update_quantization_recipe(...) rules sequentially on
the single qt object.
- Call
qt.quantize() and qt.export_model(). Output every model variation
with a detailed, unique file name following the guidelines in
file_naming.md (respecting the Tier L .tflite retention exception in
model_scale_tiers.md). Save the .json recipe for EVERY variation, no
exceptions.
- Measure the stopping metric deployment-faithfully: run each exported
.tflite under the default LiteRT Interpreter and compare outputs
against cached FP32 baseline outputs on a fixed sample set (see the
"Deployment-faithful stopping metric" rule in experiment_runner.md).
Do NOT gate commits on qt.validate() — its backend differs by recipe
(XNNPACK rejects INT4), which makes INT8 and INT4 candidates
incomparable. Reserve qt.validate(...) for per-tensor sensitivity
sweeps, strictly sequentially (one validation at a time, never
concurrent), with use_xnnpack=False on INT4-containing recipes; check
whether your installed release supports multi-metric validation or a
single metric string per call (see the validation API caveats in
experiment_runner.md). CRITICAL: Look at metrics in your script to
decide which tensors to target.
Phase 3: The Empirical Search Loop (Execution)
- Two-Phase Iterative Greedy Loop: Read
experiment_runner.md for the
exact step-by-step logic.
- Phase 0 (Free Pre-Screening): Before any validation, compute the
data-free weight kurtosis ranking (
kurtosis_screening.md) to seed
the fragile-candidate list and pre-pin obvious offenders without
spending validation budget.
- Phase 1 (Finalize Protection Set): Identify fragile layers using
sensitivity analysis (primary metric, e.g., SNR for vision, KL for
LLMs) and incrementally protect them until overall model quality is
guaranteed via the modality-appropriate stopping metric on the
output tensor. Protection mechanism priority depends on scale tier:
no_quantize skips for Tier S; 8-bit pins and algorithm upgrades
before skips for Tier M/L. Freeze the protection set once the stopping
metric is good.
- Phase 2 (Iterative Greedy INT4 Squashing): Rank remaining
unprotected layers by robustness and iteratively squash them
block-by-block (or batch-by-batch with bisection, per your tier) to
4-bit, validating the stopping metric after each step to prevent
compounding noise. Revert failing blocks and continue with the next
candidate; halt only after 3 consecutive reverts or when the
validation budget is exhausted.
- CRITICAL: When designing the Phase 2 condition, determine
threshold bounds dynamically relative to the optimal configuration
found in Phase 1. Do not use hardcoded scalar assumptions because
End-to-End metrics can output naturally large arbitrary scalars!
This applies to EVERY script in the search, including post-hoc
audit and refinement scripts.
- Phase 3 (Post-Squash Audit & Ablation — MANDATORY, budget
permitting): one cost-aware round of protection additions on the
final mixed model, followed by an ablation of any expensive
protection committed before a cheaper superseding one. Empirically
this phase recovers most of the final quality — see
experiment_runner.md.
- Algorithm Axis: If a configuration fails its quality bound, try an
algorithm upgrade (GPTQ, Hadamard rotation, MSE calibration — see
algorithms.md) on the failing layers before retreating to a higher
bit-width. For Tier L transformers, explore the algorithm axis FIRST.
- Tier L Direction Inversion (INT4-First): For Tier L models, INT4
weights are the default target, not the aggressive endpoint. Instead
of squashing down from INT8, start from an all-INT4 baseline (best
algorithm, embedding/LM head pinned to INT8) and greedily PROMOTE the
most fragile blocks up to INT8 until the stopping metric passes. See
the "Inverted Search Direction for Tier L" section of
model_scale_tiers.md.
- Loop Execution: Build out the script to execute both Phase 1
(protection set finalization via the stopping metric) and Phase 2
(iterative greedy INT4 squashing on unprotected layers) automatically.
Phase 4: Exploring the Pareto Frontier
Provide options. You must output, record, and export to disk EVERY SINGLE
recipe permutation (.json) and validation metrics file generated during your
exploration (including failed baseline attempts, intermediate steps, and the
final optimized models). Export the quantized .tflite for every permutation
on Tier S/M; on Tier L, export .tflite for the baseline and Top 3 profiles
(see model_scale_tiers.md). Do not just keep results in memory. This output
directory is necessary so the user can plot a rich Pareto curve.
Among all explored recipes, you must explicitly highlight and recommend the
Top 3 recipes in your final report, mapped to these key profiles. The
profiles are defined by the USER-FACING trade-off they optimize, not by their
construction (post-audit refinements often blur the constructive
definitions):
- Quality Profile (maximum fidelity within the size bound): Typically
remaining unprotected layers strictly at 8-bit, inheriting the finalized
protection list.
- Compact Profile (smallest model within the error tolerance):
Typically pushes remaining unprotected layers to 4-bit (with the
best-performing algorithm variant), inheriting the finalized protection
list.
- Balanced Profile (best error-per-MB knee of the frontier): Result of
differential sensitivity analysis on remaining unprotected layers
(pushing robust layers to 4-bit and moderately sensitive layers to
8-bit), inheriting the finalized protection list. Select the knee
programmatically — the admissible point preceding the largest marginal
error jump — rather than assuming the mixed recipe is automatically the
knee.
Tag the three chosen artifacts with the profile suffix from
file_naming.md §G so the recommendation is readable from the filename.
For Tier L models under the inverted INT4-first search, the same three
profiles map to: Compact = pure INT4 baseline (+ pinned embedding/LM
head), Balanced = minimal INT8 promotions needed to pass the quality
bound, and Quality = generous promotion of all flagged-fragile blocks to
INT8 (see model_scale_tiers.md).
Agent Verification Protocol (Two-Pass Loop)
To ensure code quality, verifications must follow a strict Two-Pass Recursive
Review:
- Pre-flight Self-Check: Ensure the python code parses and runs
successfully before reviewing.
- Pass 1 (Comprehensive Audit): If your harness supports subagents, you
MUST literally invoke a reviewer subagent — do not just hallucinate a
review yourself. Pass your python script implementation to the subagent
via the prompt, and instruct it to mathematically evaluate bit depths,
proper API usage (not raw JSON hacking), isolation of quantizers,
multi-metric evaluations, and adherence to the scale-tier budget. Wait for
its message back. If your harness does NOT support subagents, perform a
structured self-review: re-read your script top-to-bottom against every
item in the Reviewer checklist below, and record each pass/fail verdict
explicitly in the report.
- Pass 2 (Delta Verification): Provide the updated code for a second
review pass and verify all fixes are applied without regressions.
- If an impasse is reached, yield to human.
Role-Based Execution Guidelines
Contributor: Adhere to experiment_runner.md strictly. Prevent Data
Leakage by never hardcoding explicitly named tutorial layers in your
implementations; find and parse the actual broken scopes generically.
Maintainer: Ensure backwards compatibility when updating scripts.
Reviewer: Organize feedback into clear categories: [Quantization Quality], [LiteRT Quantizer API Health], [Experiment Rigor], [Reporting]. Reject
code that lacks multi-metric tracking or multiple Pareto options.
- Rule: You MUST reject any script that uses manual trial-and-error
regex arrays to inflate size instead of the proper Two-Phase greedy
loop.
- Rule: You MUST reject any script that doesn't explicitly program
the Greedy Phase 2 squashing loop sorting layers by robustness
(single-step or batched-with-bisection per the declared scale tier).
- Rule: You MUST reject any script that halts Phase 2 permanently on
the FIRST failed block instead of reverting-and-continuing.
- Rule: You MUST reject any script whose stopping metric ignores the
modality (e.g. output MSE used for an LLM instead of output KL).
- Rule: You MUST reject any script whose commit/revert gate mixes
validation backends across phases (e.g. XNNPACK-backed numbers for
INT8 steps vs reference-kernel numbers for INT4 steps). The gate
must be the deployment-faithful stopping metric measured on the
exported model under the default runtime.
- Rule: You MUST reject any script that ignores the scale-tier
budget (e.g. unbatched per-block validation on a Tier L model).
- Rule: You MUST reject the code if the artifact paths don't strictly
generate
_validation_metrics.json and follow the
[execution]_[precision]...[ver] schema exactly as defined in
file_naming.md and output_report.md.
Post-Execution Walkthrough Guidelines
After execution, format your final response strictly:
- Phase 1: Executive Summary & Bound Achievements: Technical TL;DR.
Provide detailed mathematical reasoning (using the selected validation
metrics) justifying your layer assignments (protected vs int8 vs int4) for
the models on the frontier. State the scale tier and the number of
validations consumed vs the budget.
- Phase 2: Architectural Footprint: Paths of generated scripts, quantized
models
.tflite files, quantization recipes .json files, Pareto graph,
and quantization_report.md.
- Phase 3: Precision & Sanity Guarantee: Confirm validations evaluated
distinct error paths jointly to decide configurations and verify data
leakage checks (generic regex abstraction used over hallucinated names;
calibration/evaluation sample disjointness if static quantization was
used).
- Phase 4: Verification Audit Trail: Explicit review pass/fail status
scorecard and delta fixes (subagent-based or structured self-review,
whichever was performed).
1---2name: auto-quant-23description: >-4 Systematically explore quantization configurations for a TFLite model using the LiteRT Quantizer API, finding the optimal recipe that balances file size and accuracy. Use this skill whenever the user wants to quantize a model, optimize a recipe, explore quantization tradeoffs, minimize size bounds, or perform selective quantization using LiteRT Quantizer framework. Applies to any model scale and modality: CNNs, segmentation nets, classifiers, embedding models, audio models, and LLMs.5metadata:6 source: https://skillmd.com/skills/google-ai-edge/auto-quant-27---89# Agent Contribution Guide: LiteRT Quantizer Recipe Exploration1011Welcome, AI Agent! This skill governs how you autonomously explore and generate12the optimal selective quantization recipe for any given LiteRT FlatBuffer13(`.tflite`) model using the LiteRT Quantizer framework. Strict adherence14to these protocols ensures code is high-quality, memory safe, and meets15verification protocols.1617Your goal is to build an iterative search loop in Python that actively18interrogates the baseline model for numerical degradation, updates the recipe on19the fly to protect highly sensitive tensors, and produces a bounding-box of20Pareto-optimal models for the user.2122## Skill Index2324Leverage these project-specific skill guides to navigate and execute tasks25effectively.2627**Core guides (read ALL of these before proceeding):**2829| File | Contents |30| --- | --- |31| [`README.md`](../../README.md) | Overview of the LiteRT Quantizer framework and the repo's structure |32| [`README.md`](../../../../README.md#operator-coverage) | What configurations are supported for each operator |33| [`model_scale_tiers.md`](model_scale_tiers.md) | Scale tier classification (S/M/L) with per-tier search budgets, batched squashing, and memory guardrails |34| [`algorithms.md`](algorithms.md) | Registry of available quantization algorithms and when to select each one |35| [`error_metric_selection.md`](error_metric_selection.md) | Registry of validation error metrics, task-specific selection, distribution-relative thresholding, and per-modality stopping metrics |36| [`file_naming.md`](file_naming.md) | Standardized conventions for naming exported models, recipes, evaluation artifacts, and validation reports |37| [`quantization_pattern.md`](quantization_pattern.md) | Empirical patterns: what works, what doesn't, and why |38| [`snr_best_practices.md`](snr_best_practices.md) | How to compute, parse, and threshold per-tensor metrics to locate fragile layers |39| [`kurtosis_screening.md`](kurtosis_screening.md) | Zero-validation, data-free layer sensitivity screening via weight kurtosis, plus the trimmed z-score outlier detection routine |40| [`regex_targeting.md`](regex_targeting.md) | How to write readable, unified block-level regex patterns to avoid scattered precision overlaps |41| [`size_estimation.md`](size_estimation.md) | How to compute theoretical compressed model size |42| [`experiment_runner.md`](experiment_runner.md) | Memory-safe experiment loop, batched squashing, the algorithm search axis |43| [`pareto_curve_plotting.md`](pareto_curve_plotting.md) | Guidelines on formatting the Pareto visualization, drawing frontiers, and handling outliers |44| [`output_report.md`](output_report.md) | How to format and organize the output produced |4546**Pluggable task evaluation family (read ONLY the file matching the model's47modality):**4849| File | Applies To |50| --- | --- |51| [`image_segmentation_eval.md`](image_segmentation_eval.md) | Segmentation / detection models (masks, mIoU/Dice, heatmaps) |52| [`llm_eval.md`](llm_eval.md) | Generative LLMs (multi-signature handling, token agreement, KL, sample generations) |53| [`classification_eval.md`](classification_eval.md) | Classifiers (Top-1/Top-5 agreement, confidence drift) |5455**Conditional guides (read when applicable):**5657| File | Read When |58| --- | --- |59| [`calibration_data.md`](calibration_data.md) | Using static quantization (calibration dataset rules, leakage prevention, manifests) |60| [`latency_benchmarking.md`](latency_benchmarking.md) | The user states a latency requirement or asks for on-device numbers |6162## Phase 0: Understand the Framework and Classify the Model6364Before beginning, explicitly use specific file reading tools to read all core65guides in the Skill Index, plus the ONE task evaluation guide matching the66model's modality, plus any applicable conditional guides.6768Then, before writing any plan:69701. **Classify the modality** (segmentation, LLM, classification, embedding,71 audio, regression) from the model's signatures, I/O tensor shapes, and the72 user's description. This selects your metrics73 (`error_metric_selection.md`) and evaluation family.742. **Classify the scale tier** (S/M/L per `model_scale_tiers.md`) from the75 float model file size and signature count. This selects your search76 budget, batching strategy, and protection-mechanism priority.7778## 1. Implementation Plan Guidelines7980Because complex selective quantization tasks can cause context loss during81execution, your first action MUST be to draft a highly explicit, self-contained82Implementation Plan. Since you will follow this plan closely, you must bake all83necessary context directly into it.8485**You MUST output this plan as an artifact file** (e.g.86`implementation_plan.md`, using your file-writing tool). In interactive87sessions, **HALT after writing the plan** and explicitly ask the user for88permission to proceed before you make *any* code edits or run *any* loops. If89the user has already provided explicit bounds AND explicitly requested90unattended/autonomous execution (e.g. a CI pipeline), you may proceed without91halting, but the plan artifact is still mandatory. You must strictly follow92this plan once approved.9394Your printed plan must contain:9596### Step 1: Context & Role Declaration9798* **Role Judgment**: Declare your role (Contributor, Maintainer, or Reviewer).99* **Model Classification**: State the detected modality, the scale tier, and100 the resulting metric selections and search budgets.101* **Context Summary**: Explicitly write out a 2-3 sentence summary of the102 specific constraints you read from the skill docs. You MUST explicitly state103 that your artifact names will obey the strict rules from `file_naming.md`104 and explicitly confirm you will produce a Pareto visualization per105 `pareto_curve_plotting.md`.106107### Step 2: Technical Design & File List108109* **Target Files**: Explicitly list all files you plan to create or modify.110 Please adhere strictly to the `output_report.md` requirements.111* **Optimization Constraints**: Briefly outline how you will handle target112 file size bounds, acceptable error thresholds across MULTIPLE metrics, and113 generic regex targeting to prevent data leakage.114115### Step 3: Experiment Harness Architecture116117* **Metric Selection**: Infer from the float model what metrics you should use118 for validation following the guidelines of `error_metric_selection.md`,119 including the modality-appropriate stopping metric.120* **Experiment Blueprint**: Explicitly outline your script structure focusing121 on proper API sequential updates, memory-safe `Quantizer` instantiation per122 permutation, strictly sequential `qt.validate()` calls (one validation at a123 time), explicit garbage collection (`import gc; gc.collect()`), and124 selecting layer candidates. **Most importantly, explicitly confirm you are125 implementing the stringent Two-Phase loop mandated by `experiment_runner.md`126 (Phase 1: Freezing protection set, Phase 2: Iterative Greedy INT4 squashing127 sorted by robustness, batched per your scale tier). Do not hallucinate128 trial-and-error hacks.**129* **Budget Declaration**: State the per-tier validation cap and sample count130 you will operate under (from `model_scale_tiers.md`).131132### Step 4: Verification Plan133134* **Review Hand-off**: Document your explicit commitment to run the Two-Pass135 Review Loop (see "Agent Verification Protocol" below) on your execution136 script *before* you present the final report. If your harness supports137 launching subagents, commit to delegating the review to a reviewer138 subagent; otherwise commit to performing the structured self-review pass139 against the Reviewer checklist. ONLY start the review after you think140 you're done with the quantization exploration process.141142### Step 5: Post-Execution Walkthrough143144* Acknowledge that the final response will adhere to the Post-Execution145 Walkthrough Guidelines (below).146147## Phase 1: Setup & Bounding Box1481491. **Ask the User for Bounds**: If the user did not specify, explicitly ask for150 either their **Minimum Model Compression Ratio / Model Size** OR their **Max151 Tolerable Error Bound**. Our objective is to find either the best quality152 model given the minimum compression ratio or the smallest model given the153 max tolerable error bound.1542. **Acquire Verification Preprocessing**: In order to use `qt.validate()`, you155 need numpy tensor inputs matching the model's signature(s). You may copy156 the input processing logic from existing manual human scripts in the157 directory (for example `processing.py`), or construct tokenized prompts158 for LLMs per `llm_eval.md`.159 * **CRITICAL CONSTRAINT (NO LEAKAGE)**: You MUST NOT copy the human's160 manual selective recipe adjustments from existing sources or tutorials161 to blindly inject layer names. You must discover which layers to162 protect dynamically based on your own validation metric parsing!163 This equally applies to the empirical anecdotes and example values164 inside these skill guides: treat them as unverified priors that165 motivate WHERE to look, never as pre-validated protection lists or166 expected metric values. Every protection you commit must be167 justified by a measurement from YOUR current run.168 * **Single-source-input warning**: If the user provides only ONE test169 input, deterministic augmentations of it are acceptable for the170 search loop, but the resulting recipe may be mildly overfit to that171 input. You MUST state this limitation explicitly in the final report172 and recommend re-validating the chosen profiles on a small held-out173 set before production use.1743. **Effort & Sizing Logic**: The user's size bounds represent a *limit*, NOT a175 *target to match exactly*. You are authorized to return models that are176 smaller than requested! The primary goal is to **maximize accuracy** (or the177 target metric) while guaranteeing the model is small enough. If you can make178 it even smaller while retaining acceptable accuracy, do so. Never179 artificially inflate the model just to hit a bound. Try multiple iterations180 until you have found the best possible quantization recipe, within the181 validation budget of your scale tier.182183## Phase 2: Building the Iteration Harness184185Do not try to guess the recipe blindly or write concurrent sweeps. You must186construct a dedicated, repeatable Python evaluation script (e.g.187`explore_litertq_model.py`) that operates on ONE recipe at a time. Do not run188parallel configurations or concurrent validations.1891901. Load the float baseline `.tflite` model and run data through preprocessing.1912. Instantiate `quantizer.Quantizer()` and load **ONE default baseline**192 starting recipe via the API.1933. Inject targeted `qt.update_quantization_recipe(...)` rules sequentially on194 the single `qt` object.1954. Call `qt.quantize()` and `qt.export_model()`. Output every model variation196 with a detailed, unique file name following the guidelines in197 `file_naming.md` (respecting the Tier L `.tflite` retention exception in198 `model_scale_tiers.md`). Save the `.json` recipe for EVERY variation, no199 exceptions.2005. Measure the **stopping metric deployment-faithfully**: run each exported201 `.tflite` under the default LiteRT `Interpreter` and compare outputs202 against cached FP32 baseline outputs on a fixed sample set (see the203 "Deployment-faithful stopping metric" rule in `experiment_runner.md`).204 Do NOT gate commits on `qt.validate()` — its backend differs by recipe205 (XNNPACK rejects INT4), which makes INT8 and INT4 candidates206 incomparable. Reserve `qt.validate(...)` for per-tensor sensitivity207 sweeps, strictly sequentially (one validation at a time, never208 concurrent), with `use_xnnpack=False` on INT4-containing recipes; check209 whether your installed release supports multi-metric validation or a210 single metric string per call (see the validation API caveats in211 `experiment_runner.md`). **CRITICAL: Look at metrics in your script to212 decide which tensors to target.**213214## Phase 3: The Empirical Search Loop (Execution)2152161. **Two-Phase Iterative Greedy Loop**: Read `experiment_runner.md` for the217 exact step-by-step logic.218 - **Phase 0 (Free Pre-Screening)**: Before any validation, compute the219 data-free weight kurtosis ranking (`kurtosis_screening.md`) to seed220 the fragile-candidate list and pre-pin obvious offenders without221 spending validation budget.222 - **Phase 1 (Finalize Protection Set)**: Identify fragile layers using223 sensitivity analysis (primary metric, e.g., SNR for vision, KL for224 LLMs) and incrementally protect them until overall model quality is225 guaranteed via the **modality-appropriate stopping metric** on the226 output tensor. Protection mechanism priority depends on scale tier:227 `no_quantize` skips for Tier S; 8-bit pins and algorithm upgrades228 before skips for Tier M/L. Freeze the protection set once the stopping229 metric is good.230 - **Phase 2 (Iterative Greedy INT4 Squashing)**: Rank remaining231 unprotected layers by robustness and iteratively squash them232 block-by-block (or batch-by-batch with bisection, per your tier) to233 4-bit, validating the stopping metric after each step to prevent234 compounding noise. **Revert failing blocks and continue with the next235 candidate; halt only after 3 consecutive reverts or when the236 validation budget is exhausted.**237 * **CRITICAL**: When designing the Phase 2 condition, determine238 threshold bounds dynamically relative to the optimal configuration239 found in Phase 1. Do not use hardcoded scalar assumptions because240 End-to-End metrics can output naturally large arbitrary scalars!241 This applies to EVERY script in the search, including post-hoc242 audit and refinement scripts.243 - **Phase 3 (Post-Squash Audit & Ablation — MANDATORY, budget244 permitting)**: one cost-aware round of protection additions on the245 final mixed model, followed by an ablation of any expensive246 protection committed before a cheaper superseding one. Empirically247 this phase recovers most of the final quality — see248 `experiment_runner.md`.249 - **Algorithm Axis**: If a configuration fails its quality bound, try an250 algorithm upgrade (GPTQ, Hadamard rotation, MSE calibration — see251 `algorithms.md`) on the failing layers before retreating to a higher252 bit-width. For Tier L transformers, explore the algorithm axis FIRST.253 - **Tier L Direction Inversion (INT4-First)**: For Tier L models, INT4254 weights are the default target, not the aggressive endpoint. Instead255 of squashing down from INT8, start from an all-INT4 baseline (best256 algorithm, embedding/LM head pinned to INT8) and greedily PROMOTE the257 most fragile blocks up to INT8 until the stopping metric passes. See258 the "Inverted Search Direction for Tier L" section of259 `model_scale_tiers.md`.2602. **Loop Execution**: Build out the script to execute both Phase 1261 (protection set finalization via the stopping metric) and Phase 2262 (iterative greedy INT4 squashing on unprotected layers) automatically.263264## Phase 4: Exploring the Pareto Frontier265266Provide options. You must output, record, and **export to disk EVERY SINGLE**267recipe permutation (`.json`) and validation metrics file generated during your268exploration (including failed baseline attempts, intermediate steps, and the269final optimized models). Export the quantized `.tflite` for every permutation270on Tier S/M; on Tier L, export `.tflite` for the baseline and Top 3 profiles271(see `model_scale_tiers.md`). Do not just keep results in memory. This output272directory is necessary so the user can plot a rich Pareto curve.273274Among all explored recipes, you must explicitly highlight and **recommend the275Top 3** recipes in your final report, mapped to these key profiles. The276profiles are defined by the USER-FACING trade-off they optimize, not by their277construction (post-audit refinements often blur the constructive278definitions):279280* **Quality Profile (maximum fidelity within the size bound)**: Typically281 remaining unprotected layers strictly at 8-bit, inheriting the finalized282 protection list.283* **Compact Profile (smallest model within the error tolerance)**:284 Typically pushes remaining unprotected layers to 4-bit (with the285 best-performing algorithm variant), inheriting the finalized protection286 list.287* **Balanced Profile (best error-per-MB knee of the frontier)**: Result of288 differential sensitivity analysis on remaining unprotected layers289 (pushing robust layers to 4-bit and moderately sensitive layers to290 8-bit), inheriting the finalized protection list. Select the knee291 programmatically — the admissible point preceding the largest marginal292 error jump — rather than assuming the mixed recipe is automatically the293 knee.294295Tag the three chosen artifacts with the profile suffix from296`file_naming.md` §G so the recommendation is readable from the filename.297298For Tier L models under the inverted INT4-first search, the same three299profiles map to: **Compact** = pure INT4 baseline (+ pinned embedding/LM300head), **Balanced** = minimal INT8 promotions needed to pass the quality301bound, and **Quality** = generous promotion of all flagged-fragile blocks to302INT8 (see `model_scale_tiers.md`).303304## Agent Verification Protocol (Two-Pass Loop)305306To ensure code quality, verifications must follow a strict Two-Pass Recursive307Review:3083091. **Pre-flight Self-Check**: Ensure the python code parses and runs310 successfully before reviewing.3112. **Pass 1 (Comprehensive Audit)**: If your harness supports subagents, you312 MUST literally invoke a reviewer subagent — do not just hallucinate a313 review yourself. Pass your python script implementation to the subagent314 via the prompt, and instruct it to mathematically evaluate bit depths,315 proper API usage (not raw JSON hacking), isolation of quantizers,316 multi-metric evaluations, and adherence to the scale-tier budget. Wait for317 its message back. If your harness does NOT support subagents, perform a318 structured self-review: re-read your script top-to-bottom against every319 item in the Reviewer checklist below, and record each pass/fail verdict320 explicitly in the report.3213. **Pass 2 (Delta Verification)**: Provide the updated code for a second322 review pass and verify all fixes are applied without regressions.3234. If an impasse is reached, yield to human.324325## Role-Based Execution Guidelines326327* **Contributor**: Adhere to `experiment_runner.md` strictly. Prevent Data328 Leakage by never hardcoding explicitly named tutorial layers in your329 implementations; find and parse the actual broken scopes generically.330* **Maintainer**: Ensure backwards compatibility when updating scripts.331* **Reviewer**: Organize feedback into clear categories: `[Quantization332 Quality]`, `[LiteRT Quantizer API Health]`, `[Experiment Rigor]`, `[Reporting]`. Reject333 code that lacks multi-metric tracking or multiple Pareto options.334335 * **Rule**: You MUST reject any script that uses manual trial-and-error336 regex arrays to inflate size instead of the proper Two-Phase greedy337 loop.338 * **Rule**: You MUST reject any script that doesn't explicitly program339 the Greedy Phase 2 squashing loop sorting layers by robustness340 (single-step or batched-with-bisection per the declared scale tier).341 * **Rule**: You MUST reject any script that halts Phase 2 permanently on342 the FIRST failed block instead of reverting-and-continuing.343 * **Rule**: You MUST reject any script whose stopping metric ignores the344 modality (e.g. output MSE used for an LLM instead of output KL).345 * **Rule**: You MUST reject any script whose commit/revert gate mixes346 validation backends across phases (e.g. XNNPACK-backed numbers for347 INT8 steps vs reference-kernel numbers for INT4 steps). The gate348 must be the deployment-faithful stopping metric measured on the349 exported model under the default runtime.350 * **Rule**: You MUST reject any script that ignores the scale-tier351 budget (e.g. unbatched per-block validation on a Tier L model).352 * **Rule**: You MUST reject the code if the artifact paths don't strictly353 generate `_validation_metrics.json` and follow the354 `[execution]_[precision]...[ver]` schema exactly as defined in355 `file_naming.md` and `output_report.md`.356357## Post-Execution Walkthrough Guidelines358359After execution, format your final response strictly:3603611. **Phase 1: Executive Summary & Bound Achievements**: Technical TL;DR.362 Provide detailed mathematical reasoning (using the selected validation363 metrics) justifying your layer assignments (protected vs int8 vs int4) for364 the models on the frontier. State the scale tier and the number of365 validations consumed vs the budget.3662. **Phase 2: Architectural Footprint**: Paths of generated scripts, quantized367 models `.tflite` files, quantization recipes `.json` files, Pareto graph,368 and `quantization_report.md`.3693. **Phase 3: Precision & Sanity Guarantee**: Confirm validations evaluated370 distinct error paths jointly to decide configurations and verify data371 leakage checks (generic regex abstraction used over hallucinated names;372 calibration/evaluation sample disjointness if static quantization was373 used).3744. **Phase 4: Verification Audit Trail**: Explicit review pass/fail status375 scorecard and delta fixes (subagent-based or structured self-review,376 whichever was performed).