Validate BSP Image
Status: the DUT-access contract is stable; the rest of the
validation procedure is a skeleton.
Purpose
Confirm that a freshly customized BSP landed correctly — both as a
static artifact on disk and as a running system on the target —
without re-promoting or re-flashing. Forms the validation tail of
Deploy in the Setup → Customize → Build → Deploy pipeline (see
../../context/bsp-customization-workflow.md
for the pipeline view) and is independently re-runnable.
Prerequisites
- Active target-platform profile with
bsp_image: resolved (run
/jetson-init-image first).
- For static-only scope: nothing further; the skill reads
<bsp_image.root_path> directly.
- For on-target scope:
/jetson-flash-image already pushed the staged BSP to the DUT.
dut_access: block authored in the active profile (or filled
interactively at runtime — see ## DUT access).
- Host tooling per transport:
ssh + sshpass for SSH; Python
3.6+ with pyserial for UART.
- Required env vars resolvable on the host when
auth=password
or sudo.method=password (password_env / sudo.password_env
name the env var; never inline the secret in YAML).
When to invoke
- After
jetson-flash-image has put the BSP onto a target.
- The user explicitly asks to validate, test, or run smoke /
regression checks on a flashed device.
- As a CI gate before declaring a customization batch shippable.
Instructions
The procedure below is a skeleton.
- Read the active target per the prerequisite contract.
- Choose validation scope — static (against
bsp_image on
disk, no DUT needed) and/or on-target (DUT must be booted
from the just-flashed image).
- Static checks (if in scope):
- Required artifacts present at expected paths in
<bsp_image.root_path>/Linux_for_Tegra/.
- DTB / module checksum or signature verification.
- Partition-layout sanity vs. the per-board
.conf's XML.
- Cross-check overlay-staged outputs against
bsp_image
(they should be identical post-promotion).
- Initramfs ↔ kernel + rootfs module coherence. Extract
<bsp_image>/Linux_for_Tegra/bootloader/l4t_initrd.img and
<bsp_image>/Linux_for_Tegra/rootfs/boot/initrd; verify
three invariants against the promoted bsp_image state:
(a) <bsp_image>/Linux_for_Tegra/kernel/Image matches
<bsp_image>/Linux_for_Tegra/rootfs/boot/Image byte-for-
byte — drift here means
the "Mirror kernel Image into rootfs" step
was skipped and any subsequent initramfs refresh built
against the previous kernel.
(b) for each module path the two initrd images ship, the
bytes / md5 match the file under
<bsp_image>/Linux_for_Tegra/rootfs/lib/modules/<ver>/ —
any drift means modules will be shadowed at early boot.
(c) the vermagic stamped on every initramfs *.ko matches
the UTS_RELEASE reachable from
<bsp_image>/Linux_for_Tegra/kernel/Image (e.g. parse the
Linux version … string with strings) — a vermagic skew
means the kernel Image was refreshed without rerunning
l4t_update_initrd.sh, and modules will fail to load with
"disagrees about version of symbol …". All three failure
modes are closed by /jetson-promote-image's gate on
either kernel/Image or rootfs/lib/modules/ plus the
kernel-Image mirror; surface drift here and route the user
back to a clean promote. See
../jetson-promote-image/SKILL.md.
- On-target checks (if in scope):
- Connect to the DUT per the
## DUT access
section below — resolve transport (ssh / uart), credentials,
and sudo method from the active profile's dut_access:
block (with interactive fallback when fields are missing /
marked prompt), then run the connection probe and refuse
if it fails.
- Confirm boot reached userspace.
- Run the selected test suite (smoke, regression, focused
per-customization, ad-hoc).
- Loaded-module srcversion drift. For modules a
customization is known to have rebuilt, compare
cat /sys/module/<name>/srcversion on the DUT against the
modinfo /lib/modules/$(uname -r)/.../<name>.ko | awk '/srcversion/ {print $2}' reading. A mismatch means the
kernel is running an older copy than the rootfs ships —
almost always a stale initramfs (the bootloader-side initrd
shipped a pre-customize module, it loaded first, and the
rootfs copy cannot replace a live module). Recommend
re-running /jetson-promote-image and re-flashing. Note:
some modules don't emit srcversion; fall back to an md5
check against the binary the kernel loaded by extracting the
region under /sys/module/<name>/sections/ or by comparing
behaviorally (printk / sysfs nodes / DT properties the new
version is known to expose).
- Running kernel vs. rootfs kernel
Image. Compare
cat /proc/version (or uname -v) on the DUT against the
Linux version … string extracted from
/boot/Image (strings /boot/Image | grep -m1 'Linux version').
A mismatch — usually the build timestamp / LOCALVERSION —
means the bootloader is running an older kernel Image than
the rootfs holds, almost certainly because a fresh Image
was promoted but the initramfs / extlinux.conf / QSPI
boot partition wasn't refreshed. Modules in the rootfs
will then have a different vermagic and any subsequent
modprobe of a built-against-the-new-kernel .ko will
fail. Recommend /jetson-promote-image and re-flash.
- Userspace dmesg readability. Ubuntu 22.04 sets
kernel.dmesg_restrict=1; non-root dmesg reads return
"Operation not permitted" and silently zero hits. Every
dmesg-based check must run with sudo (or temporarily
lower the restriction via sudo sysctl kernel.dmesg_restrict=0). Surfacing this in the validate
layer keeps printk-based customization checks honest.
- Collect results, logs, artifacts.
- Summary: per-check pass/fail, overall verdict, where
logs and artifacts landed.
DUT access
The on-target leg needs a way to reach the just-flashed DUT.
Two transports are supported as full peers: ssh (primary) and
uart (fallback for DUTs with no network).
The contract is locked in but lives in
references/dut-access.md to keep this
SKILL.md under the agent-routing budget. That reference covers:
- The
dut_access: profile schema (ssh, uart, sudo, workdir).
- Resolution order (profile → env var → interactive prompt) with
the full refusal-trigger table.
- The mandatory connection probe (
uname -r +
cat /etc/nv_tegra_release) and its output-validation rules.
- File transfer per transport (
scp vs. base64-over-tty, with
the >100 KB warning).
- Sudo invocation matrix per
sudo.method × transport.
- UART implementation contract that
scripts/uart_session.py
honors (state machines for login / exec / push / pull, exit
codes, robustness notes).
- Security notes (password handling, host-key pinning).
Available Scripts
| Script |
Purpose |
Arguments |
scripts/uart_session.py |
UART transport black box for the on-target leg: login, command exec (with optional sudo), and base64-over-tty file transfer. Replaces ssh when the DUT has no network or ssh is broken. |
--tty <dev> [--baud <n>] --user <name> --password-env <ENVVAR> [--sudo-password-env <ENVVAR>] [--shell-prompt <regex>] [--lock-strategy refuse|wait] <probe|exec|push|pull> [...] |
Invocation (the skill calls scripts as a subprocess — run_script()
in agent-runtime terminology):
# run_script: probe the DUT over UART
DUT_UART_PASSWORD_ENV=DUT_UART_PWD \
DUT_UART_PWD="$(read -rs -p 'UART login pw: '; echo "$REPLY")" \
scripts/uart_session.py \
--tty /dev/ttyACM0 --baud 115200 \
--user ubuntu --password-env DUT_UART_PASSWORD_ENV \
probe
# run_script: exec a sudo command and capture exit code
scripts/uart_session.py --tty /dev/ttyACM0 --user ubuntu \
--password-env DUT_UART_PASSWORD_ENV \
--sudo-password-env DUT_SUDO_PWD \
exec --use-sudo 'dmesg | tail -200'
The script's exit code is the contract — see the exit-code table in
UART implementation contract.
Examples
Static-only validation (no DUT needed):
/jetson-validate-image
> static checks only against the staged BSP
On-target validation over SSH after a freshly flashed DUT:
/jetson-flash-image
↓
/jetson-validate-image
> on-target checks via dut_access.ssh
On-target validation over UART (no network on the DUT):
/jetson-validate-image
> use the uart transport at /dev/ttyACM0; the dut_access.uart block
in the profile already has the tty and login_password_env wired up
Limitations
- Placeholder skill — only the DUT-access contract and the
uart_session.py helper are locked in. The static-check list,
test-suite selection, result-sink layout, and pass/fail policy
are tracked under ## Open items and may change.
- UART file transfer is byte-banged base64 at ~10 KB/s on 115200
baud — emits a warning for sources > 100 KB but proceeds. For
high-volume transfers, switch to the SSH transport.
uart_session.py opens and closes the tty per subcommand
invocation (~1–2 s login per call). Validation passes running
10 commands amortize poorly on UART; prefer SSH.
- SSH uses
StrictHostKeyChecking=accept-new with a per-profile
known_hosts file. A fingerprint change refuses — typically
means the DUT was reflashed (host keys regenerated) or the IP
was reassigned. Remove the per-profile entry manually rather
than auto-accepting.
- Passwords are never inlined in the profile YAML — only
password_env (env-var name) is persisted. auth=prompt /
sudo.method=prompt exposes the password in the conversation
log, which is the user's responsibility to manage.
transport=uart with lock_strategy=steal is not implemented
(would require sending control characters that could corrupt the
holder's state).
Troubleshooting
| Error |
Cause |
Solution |
no DUT transport configured (refuse) |
On-target scope requested but dut_access.transport unresolved across profile / env / interactive prompt |
Author dut_access: in the active profile, or rerun with static-only scope. |
$PASSWORD_ENV is unset (refuse) |
auth=password (or sudo.method=password) names an env var that isn't exported on the host |
export <ENVVAR>=... before invoking the skill; do NOT inline the password in YAML. |
tty held by another process (refuse, exit 3) |
lock_strategy=refuse and fuser reports another holder (minicom, picocom, getty) |
Close the holding process (sudo fuser -k <tty> only if you know what's there), or rerun with lock_strategy=wait. |
pyserial import failed (exit 4) |
UART transport selected but pyserial not installed in the skill's Python |
apt install python3-serial or pip install pyserial. |
| SSH fingerprint refused |
DUT's host keys changed since the per-profile known_hosts was pinned (typically a reflash) |
Remove the matching line from <workspace>/target-platform/<profile-stem>.known_hosts and rerun — the new key will be accepted on first connect. |
login failed (exit 128) |
UART probe couldn't match a login or shell prompt within timeout |
Confirm the DUT is powered + booted to userspace; check --shell-prompt override if the DUT carries an unusual PS1; verify the --user matches a real account. |
DUT not booted from the just-flashed BSP (warn / refuse) |
/etc/nv_tegra_release on the DUT doesn't match bsp_image.version |
Re-run /jetson-flash-image and confirm the DUT actually power-cycled into the new BSP, not the previous one. |
sudo prompt detected but no sudo password configured (exit 130) |
--use-sudo passed to uart_session.py exec but --sudo-password-env not set, and DUT user lacks NOPASSWD |
Set sudo.method=password + sudo.password_env in the profile, or grant NOPASSWD on the DUT via /etc/sudoers.d/. |
command timed out (exit 129) |
Long-running DUT command exceeded the script's timeout |
Run the command directly via ssh transport, or split into shorter steps. |
References
1---2name: jetson-validate-image3description: Use after jetson-flash-image to run static BSP checks, on-target smoke/regression tests on a flashed DUT, or both. Not for build or flash steps. Triggers: validate bsp, on-target validation.4license: Apache-2.05---67# Validate BSP Image89> **Status:** the DUT-access contract is stable; the rest of the10> validation procedure is a skeleton.1112## Purpose1314Confirm that a freshly customized BSP landed correctly — both as a15static artifact on disk and as a running system on the target —16without re-promoting or re-flashing. Forms the **validation tail of17Deploy** in the Setup → Customize → Build → Deploy pipeline (see18[`../../context/bsp-customization-workflow.md`](../../context/bsp-customization-workflow.md)19for the pipeline view) and is independently re-runnable.2021## Prerequisites2223- Active target-platform profile with `bsp_image:` resolved (run24 `/jetson-init-image` first).25- For static-only scope: nothing further; the skill reads26 `<bsp_image.root_path>` directly.27- For on-target scope:28 - `/jetson-flash-image` already pushed the staged BSP to the DUT.29 - `dut_access:` block authored in the active profile (or filled30 interactively at runtime — see [`## DUT access`](#dut-access)).31 - Host tooling per transport: `ssh` + `sshpass` for SSH; Python32 3.6+ with `pyserial` for UART.33 - Required env vars resolvable on the host when `auth=password`34 or `sudo.method=password` (`password_env` / `sudo.password_env`35 name the env var; never inline the secret in YAML).3637## When to invoke3839- After `jetson-flash-image` has put the BSP onto a target.40- The user explicitly asks to validate, test, or run smoke /41 regression checks on a flashed device.42- As a CI gate before declaring a customization batch shippable.4344## Instructions4546The procedure below is a skeleton.47481. **Read the active target** per the prerequisite contract.492. **Choose validation scope** — static (against `bsp_image` on50 disk, no DUT needed) and/or on-target (DUT must be booted51 from the just-flashed image).523. **Static checks** (if in scope):53 - Required artifacts present at expected paths in54 `<bsp_image.root_path>/Linux_for_Tegra/`.55 - DTB / module checksum or signature verification.56 - Partition-layout sanity vs. the per-board `.conf`'s XML.57 - Cross-check overlay-staged outputs against `bsp_image`58 (they should be identical post-promotion).59 - **Initramfs ↔ kernel + rootfs module coherence.** Extract60 `<bsp_image>/Linux_for_Tegra/bootloader/l4t_initrd.img` and61 `<bsp_image>/Linux_for_Tegra/rootfs/boot/initrd`; verify62 three invariants against the *promoted* `bsp_image` state:63 (a) `<bsp_image>/Linux_for_Tegra/kernel/Image` matches64 `<bsp_image>/Linux_for_Tegra/rootfs/boot/Image` byte-for-65 byte — drift here means66 [the "Mirror kernel Image into rootfs" step](../jetson-promote-image/SKILL.md#mirror-kernel-image-into-rootfs-when-kernel-changed)67 was skipped and any subsequent initramfs refresh built68 against the previous kernel.69 (b) for each module path the two initrd images ship, the70 bytes / md5 match the file under71 `<bsp_image>/Linux_for_Tegra/rootfs/lib/modules/<ver>/` —72 any drift means modules will be shadowed at early boot.73 (c) the vermagic stamped on every initramfs `*.ko` matches74 the `UTS_RELEASE` reachable from75 `<bsp_image>/Linux_for_Tegra/kernel/Image` (e.g. parse the76 `Linux version …` string with `strings`) — a vermagic skew77 means the kernel `Image` was refreshed without rerunning78 `l4t_update_initrd.sh`, and modules will fail to load with79 "disagrees about version of symbol …". All three failure80 modes are closed by `/jetson-promote-image`'s gate on81 either `kernel/Image` or `rootfs/lib/modules/` plus the82 kernel-Image mirror; surface drift here and route the user83 back to a clean promote. See84 [`../jetson-promote-image/SKILL.md`](../jetson-promote-image/SKILL.md#refresh-initramfs-when-kernel-or-modules-changed).854. **On-target checks** (if in scope):86 - **Connect to the DUT** per the [`## DUT access`](#dut-access)87 section below — resolve transport (ssh / uart), credentials,88 and sudo method from the active profile's `dut_access:`89 block (with interactive fallback when fields are missing /90 marked `prompt`), then run the connection probe and refuse91 if it fails.92 - Confirm boot reached userspace.93 - Run the selected test suite (smoke, regression, focused94 per-customization, ad-hoc).95 - **Loaded-module srcversion drift.** For modules a96 customization is known to have rebuilt, compare97 `cat /sys/module/<name>/srcversion` on the DUT against the98 `modinfo /lib/modules/$(uname -r)/.../<name>.ko | awk99 '/srcversion/ {print $2}'` reading. A mismatch means the100 kernel is running an older copy than the rootfs ships —101 almost always a stale initramfs (the bootloader-side initrd102 shipped a pre-customize module, it loaded first, and the103 rootfs copy cannot replace a live module). Recommend104 re-running `/jetson-promote-image` and re-flashing. Note:105 some modules don't emit `srcversion`; fall back to an md5106 check against the binary the kernel loaded by extracting the107 region under `/sys/module/<name>/sections/` or by comparing108 behaviorally (printk / sysfs nodes / DT properties the new109 version is known to expose).110 - **Running kernel vs. rootfs kernel `Image`.** Compare111 `cat /proc/version` (or `uname -v`) on the DUT against the112 `Linux version …` string extracted from113 `/boot/Image` (`strings /boot/Image | grep -m1 'Linux version'`).114 A mismatch — usually the build timestamp / `LOCALVERSION` —115 means the bootloader is running an older kernel `Image` than116 the rootfs holds, almost certainly because a fresh `Image`117 was promoted but the initramfs / `extlinux.conf` / QSPI118 boot partition wasn't refreshed. Modules in the rootfs119 will then have a different vermagic and any subsequent120 `modprobe` of a built-against-the-new-kernel `.ko` will121 fail. Recommend `/jetson-promote-image` and re-flash.122 - **Userspace dmesg readability.** Ubuntu 22.04 sets123 `kernel.dmesg_restrict=1`; non-root `dmesg` reads return124 "Operation not permitted" and silently zero hits. Every125 `dmesg`-based check must run with `sudo` (or temporarily126 lower the restriction via `sudo sysctl127 kernel.dmesg_restrict=0`). Surfacing this in the validate128 layer keeps printk-based customization checks honest.129 - Collect results, logs, artifacts.1305. **Summary**: per-check pass/fail, overall verdict, where131 logs and artifacts landed.132133## DUT access134135The on-target leg needs a way to reach the just-flashed DUT.136Two transports are supported as full peers: `ssh` (primary) and137`uart` (fallback for DUTs with no network).138139The contract is locked in but lives in140[`references/dut-access.md`](references/dut-access.md) to keep this141SKILL.md under the agent-routing budget. That reference covers:142143- The `dut_access:` profile schema (ssh, uart, sudo, workdir).144- Resolution order (profile → env var → interactive prompt) with145 the full refusal-trigger table.146- The mandatory connection probe (`uname -r` +147 `cat /etc/nv_tegra_release`) and its output-validation rules.148- File transfer per transport (`scp` vs. base64-over-tty, with149 the >100 KB warning).150- Sudo invocation matrix per `sudo.method` × transport.151- UART implementation contract that `scripts/uart_session.py`152 honors (state machines for login / exec / push / pull, exit153 codes, robustness notes).154- Security notes (password handling, host-key pinning).155156## Available Scripts157158| Script | Purpose | Arguments |159|---|---|---|160| `scripts/uart_session.py` | UART transport black box for the on-target leg: login, command exec (with optional sudo), and base64-over-tty file transfer. Replaces ssh when the DUT has no network or ssh is broken. | `--tty <dev> [--baud <n>] --user <name> --password-env <ENVVAR> [--sudo-password-env <ENVVAR>] [--shell-prompt <regex>] [--lock-strategy refuse\|wait] <probe\|exec\|push\|pull> [...]` |161162Invocation (the skill calls scripts as a subprocess — `run_script()`163in agent-runtime terminology):164165```bash166# run_script: probe the DUT over UART167DUT_UART_PASSWORD_ENV=DUT_UART_PWD \168DUT_UART_PWD="$(read -rs -p 'UART login pw: '; echo "$REPLY")" \169 scripts/uart_session.py \170 --tty /dev/ttyACM0 --baud 115200 \171 --user ubuntu --password-env DUT_UART_PASSWORD_ENV \172 probe173174# run_script: exec a sudo command and capture exit code175scripts/uart_session.py --tty /dev/ttyACM0 --user ubuntu \176 --password-env DUT_UART_PASSWORD_ENV \177 --sudo-password-env DUT_SUDO_PWD \178 exec --use-sudo 'dmesg | tail -200'179```180181The script's exit code is the contract — see the exit-code table in182[`UART implementation contract`](references/dut-access.md#uart-implementation-contract).183184## Examples185186Static-only validation (no DUT needed):187188```189/jetson-validate-image190> static checks only against the staged BSP191```192193On-target validation over SSH after a freshly flashed DUT:194195```196/jetson-flash-image197 ↓198/jetson-validate-image199> on-target checks via dut_access.ssh200```201202On-target validation over UART (no network on the DUT):203204```205/jetson-validate-image206> use the uart transport at /dev/ttyACM0; the dut_access.uart block207 in the profile already has the tty and login_password_env wired up208```209210## Limitations211212- Placeholder skill — only the DUT-access contract and the213 `uart_session.py` helper are locked in. The static-check list,214 test-suite selection, result-sink layout, and pass/fail policy215 are tracked under `## Open items` and may change.216- UART file transfer is byte-banged base64 at ~10 KB/s on 115200217 baud — emits a warning for sources > 100 KB but proceeds. For218 high-volume transfers, switch to the SSH transport.219- `uart_session.py` opens and closes the tty per subcommand220 invocation (~1–2 s login per call). Validation passes running221 > 10 commands amortize poorly on UART; prefer SSH.222- SSH uses `StrictHostKeyChecking=accept-new` with a per-profile223 `known_hosts` file. A fingerprint change refuses — typically224 means the DUT was reflashed (host keys regenerated) or the IP225 was reassigned. Remove the per-profile entry manually rather226 than auto-accepting.227- Passwords are never inlined in the profile YAML — only228 `password_env` (env-var name) is persisted. `auth=prompt` /229 `sudo.method=prompt` exposes the password in the conversation230 log, which is the user's responsibility to manage.231- `transport=uart` with `lock_strategy=steal` is **not implemented**232 (would require sending control characters that could corrupt the233 holder's state).234235## Troubleshooting236237| Error | Cause | Solution |238|---|---|---|239| `no DUT transport configured` (refuse) | On-target scope requested but `dut_access.transport` unresolved across profile / env / interactive prompt | Author `dut_access:` in the active profile, or rerun with static-only scope. |240| `$PASSWORD_ENV is unset` (refuse) | `auth=password` (or `sudo.method=password`) names an env var that isn't exported on the host | `export <ENVVAR>=...` before invoking the skill; do NOT inline the password in YAML. |241| `tty held by another process` (refuse, exit 3) | `lock_strategy=refuse` and `fuser` reports another holder (minicom, picocom, getty) | Close the holding process (`sudo fuser -k <tty>` only if you know what's there), or rerun with `lock_strategy=wait`. |242| `pyserial import failed` (exit 4) | UART transport selected but `pyserial` not installed in the skill's Python | `apt install python3-serial` or `pip install pyserial`. |243| SSH fingerprint refused | DUT's host keys changed since the per-profile `known_hosts` was pinned (typically a reflash) | Remove the matching line from `<workspace>/target-platform/<profile-stem>.known_hosts` and rerun — the new key will be accepted on first connect. |244| `login failed` (exit 128) | UART probe couldn't match a login or shell prompt within timeout | Confirm the DUT is powered + booted to userspace; check `--shell-prompt` override if the DUT carries an unusual `PS1`; verify the `--user` matches a real account. |245| `DUT not booted from the just-flashed BSP` (warn / refuse) | `/etc/nv_tegra_release` on the DUT doesn't match `bsp_image.version` | Re-run `/jetson-flash-image` and confirm the DUT actually power-cycled into the new BSP, not the previous one. |246| `sudo prompt detected but no sudo password configured` (exit 130) | `--use-sudo` passed to `uart_session.py exec` but `--sudo-password-env` not set, and DUT user lacks NOPASSWD | Set `sudo.method=password` + `sudo.password_env` in the profile, or grant NOPASSWD on the DUT via `/etc/sudoers.d/`. |247| `command timed out` (exit 129) | Long-running DUT command exceeded the script's timeout | Run the command directly via `ssh` transport, or split into shorter steps. |248249## References250251- [`references/dut-access.md`](references/dut-access.md) — full DUT-access contract: profile schema, resolution order, probe, file transfer, sudo, UART implementation, security.252- [`../../context/target-platform-contract.md`](../../context/target-platform-contract.md) — target-platform contract.253- [`../../context/bsp-customization-workflow.md`](../../context/bsp-customization-workflow.md#workflow-invariants) — Workspace edit protocol (this skill is the Deploy tail).254- [`../jetson-build-source/SKILL.md`](../jetson-build-source/SKILL.md) — Build builder; produces the artifacts this skill validates.