Skill

smolBSD — Complete Platform Reference

This skill provides exhaustive knowledge of the entire smolBSD framework — enough for an agent to understand, navigate, extend, and debug every aspect of the project.

Read §20 (Debugging Playbook) before touching anything related to image minimization, sailor, or login/PAM in a stripped image — these are the areas that have produced the most subtle real-world failures and are easy to misdiagnose from source alone.

1. Project Overview

smolBSD builds minimal, fast-booting NetBSD virtual machines (microVMs). Key properties:

• ~10 ms boot via PVH (PVHv2) on QEMU microvm machine type
• No prior NetBSD installation required on the host
• Immutable by design — images are built once, booted many times
• Host platforms: GNU/Linux, NetBSD, macOS (x86 VT-capable or ARM64 CPU recommended)
• Guest architectures: amd64, i386, evbarm-aarch64
• VMMs: QEMU (primary), Firecracker, Bhyve (BIOS mode)
• Images are raw .img disk files with FFS (NetBSD) or ext2 (Linux build hosts)

The fundamental unit is a service — a directory containing:

• NetBSD set selection (base, etc, comp, man, rescue, …)
• Build-time scripts (postinst/*.sh)
• Runtime init script (etc/rc)
• Build configuration (options.mk)

2. Project Directory Layout

smolBSD/
├── Makefile              # Entry point for manual image building (bmake)
├── mkimg.sh              # Image creation script (called by Makefile, or directly via `bmake SERVICE=<name> base`)
├── startnb.sh            # Low-level QEMU VM launcher
├── smoler.sh             # High-level CLI dispatcher: build|run|push|pull|images
├── batch.sh              # Batch launcher: N copies of a service on shifted ports
├── smoler/
│   ├── build.sh          # SMOLerfile parser → generates service dir + calls bmake
│   └── img.sh            # OCI push/pull/list (oras wrapper)
├── scripts/
│   ├── app-run.sh        # Launcher helper for the app/ GUI
│   ├── fetch.sh          # Smart curl wrapper (globbing, fresh checks)
│   ├── freshchk.sh       # Freshness check (remote Last-Modified cache in db/)
│   ├── sh                # Static shell used as /rescue/sh in rescue images
│   └── uname.sh          # Architecture/machine detection helper
├── sailor/               # Cloned sailor repo (minimization), invoked by mkimg.sh
├── service/              # All service definitions
│   ├── common/           # Shared runtime scripts bundled into /etc/include/ in VM
│   │   ├── basicrc       # Standard env, networking, devices, SSL_CERT_FILE, rc.pre/rc.local
│   │   ├── choupi        # Emoji/ASCII toggle for terminal output
│   │   ├── funcs         # rsynclite() helper (tar-based directory sync)
│   │   ├── vars          # BASEPATH, DRIVE2 path constants
│   │   ├── shutdown      # Clean halt (sync, umount, optional viocon kill signal)
│   │   ├── mount9p       # 9P filesystem mount (host directory sharing)
│   │   ├── qemufwcfg     # QEMU fw_cfg variable loader
│   │   ├── pkgin         # Package manager bootstrapper
│   │   └── sailor.vars   # Sailor integration variables
│   ├── base/             # Full base+etc system with ksh (builder image base)
│   ├── build/            # Builder microVM service (orchestrates service builds)
│   ├── rescue/           # ~10 MB minimal rescue shell
│   └── <service>/        # One directory per service
│       ├── etc/rc        # Runtime init script (MANDATORY for init(8) services)
│       ├── postinst/     # Build-time scripts executed on host/VM builder
│       ├── options.mk    # Service build variables (IMGSIZE, ADDPKGS, SETS, etc.)
│       ├── own.mk        # User overrides (git-ignored, not committed)
│       ├── sailor.conf   # Sailor minimization rules
│       ├── packages/     # Pre-built binary packages for offline install
│       └── NETBSD_ONLY   # Marker: build only on native NetBSD
├── smolerfiles/          # SMOLerfile / Dockerfile examples
│   ├── Dockerfile.inc    # Shared INCLUDE snippets
│   ├── Dockerfile.<name> # Per-service SMOLerfile (Dockerfile-compatible)
│   ├── SMOLerfile.<name> # Named SMOLerfiles (same syntax, different naming)
│   ├── *.smol            # Minimal SMOLerfiles (service name from filename)
│   └── *.inc             # Shared include fragments
├── etc/                  # VM config files for startnb.sh (-f flag)
│   └── <service>.conf    # hostfwd, imgtag, use_pty, KERNEL, NBIMG, etc.
├── bios/                 # BIOS firmware files for microvm machine type
├── confkerndev/          # Kernel driver disabler tool (SMOLIFY)
├── app/                  # Flask-based web GUI for VM management
├── www/                  # Project website and assets
├── k8s/                  # Kubernetes device plugin / deployment examples
├── misc/                 # Miscellaneous documentation
├── contribs/             # Contributed scripts
├── share/                # Shared assets (e.g. ssh.pub keys)
├── .github/workflows/    # CI/CD pipeline
│   ├── main.yml          # Builder + rescue images for amd64 + evbarm-aarch64 on push
│   └── smoler.yml        # SMOLerfile service images on smolerfiles/* push
├── db/                   # Fetch freshness cache (remote Last-Modified, see freshchk.sh)
├── images/               # Built .img disk images (empty in repo, populated at build)
├── kernels/              # Downloaded kernels (empty in repo, populated at build)
├── sets/                 # Downloaded NetBSD sets (empty in repo, populated at build)
├── pkgs/                 # Optional pre-fetched packages (empty in repo, populated at build)
├── mnt/                  # Build-time mount point (empty directory)
└── disks/                # Additional disk images

3. Two Workflows

3.1 smoler.sh (Docker-style, high-level)

smoler.sh is a thin dispatcher that routes subcommands to dedicated scripts:

smoler.sh run name resolution:

1. Strips -amd64:… or -evbarm-aarch64:… suffix to get base service name (note: an -i386:… suffix is not stripped — the etc/<base>.conf lookup for i386 images is a known quirk; the images/<image>.img fallback still works with the full name)
2. Checks for etc/<base>.conf → passes -f etc/<base>.conf to startnb.sh
3. Falls back to checking images/<image>.img → passes -i <image> to startnb.sh
4. If neither exists, shows startnb.sh -h usage

smoler.sh build regeneration: if service/<name>/ already exists, build.sh deletes its etc/rc, options.mk, postinst/ and etc/<name>.conf before regenerating them. Untracked files in the service dir (e.g. sailor.conf, own.mk) survive, but anything hand-edited in the deleted files is lost — commit what matters first.

3.2 bmake / make (Manual, low-level)

Platform-specific builder image behavior (Makefile build target):

• On NetBSD/FreeBSD/Linux: builds the builder image natively (bmake buildimg)
• On macOS (and any other OS): fetches the pre-built builder image from GitHub (bmake fetchimg); running buildimg directly on macOS fails because mkimg.sh rejects macOS
• Builder image freshness is checked via SHA256; rebuilds/fetches only when the remote changes

4. SMOLerfile / Dockerfile Reference

SMOLerfiles are nearly 100% Dockerfile-compatible. smoler/build.sh parses them line-by-line and generates:

• service/<name>/options.mk — build variables
• service/<name>/etc/rc — runtime init script
• service/<name>/postinst/postinst-N.sh — build-time execution scripts
• etc/<name>.conf — VM config for startnb.sh

4.1 Parsing Flow (build.sh internals)

1. INCLUDE expansion: INCLUDE <file> directives are resolved first by catting the referenced file inline, producing a flat temporary SMOLerfile
2. LABEL extraction: All LABEL lines (with or without smolbsd. prefix) are extracted via sed/awk, uppercased, and written to options.mk
3. Service name: From LABEL smolbsd.service=NAME, or from .smol filename (SMOLerfile.foo → SERVICE=foo)
4. Postinst-0.sh: Generated with chroot setup (pkgin bootstrap, resolv.conf, openssl certs)
5. Line-by-line parsing: Each directive generates shell commands appended to postinst scripts or etc/rc
6. Finalization: etc/rc gets . /etc/include/shutdown appended; etc/<name>.conf gets imgtag and use_pty
7. Build: Calls make (NetBSD) or bmake (elsewhere) with SERVICE=<name> IMGTAG=:<tag> build

4.2 All Supported Directives

4.3 FROM — Set Selection Details

FROM base,etc                    # Standard: base system + /etc config files
FROM base,etc,man,comp           # Full: adds man pages and compiler toolchain
FROM comp:/usr/bin/strip         # Partial: only extract /usr/bin/strip from comp set
FROM comp:/usr/libexec/*         # Glob: extract matching files from comp set
FROM base-amd64.img              # Inherit from a pre-built image

Valid set names: base, etc, man, comp, rescue, games, modules, tests, text, xbase, xcomp, xetc, xfont, xserver.

4.4 RUN — Heredoc Support

RUN <<EOF
hostname myhost
ulimit -n 4096
echo 'eval \$(resize)' >> /etc/rc.local
EOF

The parser detects <<EOF (or any tag) and appends lines until the closing tag. Quotes around the tag are stripped. Heredoc content is escaped (" → \") before being wrapped in chroot . su ${USER} -c "...".

4.5 COPY / ADD — Options

COPY --chown=clawd --chmod=600 /host/ssh.pub /home/clawd/.ssh/authorized_keys
ADD --exclude=.git ./src /app

• --chown=user:group or --chown=user — set ownership via chown -R in chroot
• --chmod=mode — set permissions via chmod -R in chroot
• --exclude=pattern — passed to rsynclite (tar-based sync)
• HTTP(S) URLs in ADD/COPY are fetched via ftp -o
• Destination paths starting with $ are treated as variable references

4.6 Generated etc/.conf Format

hostfwd=::8881-:8880,::2289-:22
imgtag=latest
use_pty=y
share=/host/path      # from VOLUME

4.7 Postinst Script Numbering

The parser generates numbered postinst scripts:

• postinst-0.sh — chroot bootstrap (pkgin setup, resolv.conf, openssl certs)
• postinst-1.sh — first RUN/COPY/ADD/USER/VOLUME/WORKDIR block (default shell)
• postinst-N.sh — new script created when SHELL directive changes the shell
• postinst.args — accumulated ARG/ENV exports shared across scripts

4.8 File Naming Conventions

• Dockerfile.<name> — standard Dockerfile naming; service name from LABEL smolbsd.service
• SMOLerfile.<name> — same syntax; service name from LABEL smolbsd.service
• <name>.smol — minimal files; service name extracted from filename itself
• *.inc — include fragments (used with INCLUDE directive)

4.9 make vs bmake

bmake is the host-side build tool (required on Linux/macOS; on NetBSD it's synonymous with make). It invokes the top-level Makefile targets (build, buildimg, base, …).

Inside the builder VM (i.e., in RUN directives and postinst/*.sh scripts), the environment is NetBSD — use plain make, not bmake. The builder VM includes make from the comp set; bmake is not guaranteed to be available.

# Wrong (bmake is a host tool, not inside the VM):
RUN cd /tmp/src && bmake && bmake install

# Correct (plain make inside the NetBSD builder VM):
RUN cd /tmp/src && make && make install

5. Service Directory Manual Reference

5.1 options.mk — All Known Variables

Conditional variables (Makefile syntax in options.mk):

.if defined(MINIMIZE) && ${MINIMIZE} == y
ADDPKGS=pkgin pkg_tarup pkg_install sqlite3 rsync curl
.endif

5.2 etc/rc — Runtime Init Script

This is the heart of every service. Standard structure:

#!/bin/sh

. /etc/include/basicrc          # Mandatory: env, networking, devices
. /etc/include/mount9p          # Optional: host directory sharing

# tmpfs mounts for writable overlays
mount -t tmpfs -o -s10M tmpfs /tmp
mount -t tmpfs -o -s10M tmpfs /var/log
mount -t tmpfs -o -s1M tmpfs /var/run
mount -t tmpfs -o -s10M -o union tmpfs /etc

# Service-specific setup (users, permissions, config)
useradd -m sshd
mkdir -p /home/sshd/.ssh

# Start services
/etc/rc.d/sshd onestart

# Main command (blocks until service exits)
exec myapp

. /etc/include/shutdown         # Clean halt

Key hooks in basicrc:

• /etc/rc.pre — custom pre-boot hook (sourced before device setup)
• /etc/rc.local — custom post-boot hook (sourced after networking, before MOUNTRO)
• SSL_CERT_FILE env var — if set, copies custom SSL certs and runs certctl rehash

5.3 postinst/*.sh — Build-Time Scripts

These execute on the build host (or builder VM) inside the mounted image root. Use for:

• Downloading external binaries with curl or ftp
• Extracting archives
• Setting up chroot environment
• Pre-configuration that doesn't need pkgin

They are NOT run inside the microVM at boot time.

Important conventions:

• Scripts run from the mounted image root (i.e., pwd is the fake root)
• Paths like etc/ssh/ refer to the image's /etc/ssh/
• Use ../service/<name>/etc/ to access files from the service directory
• Source ../service/common/funcs for rsynclite() and ../service/common/choupi for emoji output
• Check /BUILDIMG marker file to verify running inside the builder VM

5.4 own.mk — User Overrides

Not committed to git. Same format as options.mk. Loaded after options.mk so it overrides. Use for personal dev settings.

# service/myapp/own.mk (git-ignored)
IMGSIZE=1024
ADDPKGS=pkgin curl vim

5.5 sailor.vars (in service/common/)

Seed variables consumed by sailor when mkimg.sh runs minimization (see §13.2). Current contents:

• shippath — the smolBSD build drive path (/drive2), where sailor finds the image being minimized
• shipbins — baseline list of binaries always kept (init, mount, sh, useradd, login, /usr/lib/security/*, …)
• sync_dirs — directories kept in sync rather than stripped (/etc, certs, pkgin config, terminfo, zoneinfo, /var/log)
• packages — package names treated as ship targets (curl, rsync)

Relationship: treat sailor.vars as the floor, and per-service sailor.conf as the diff on top of it. If a stripped image later fails in surprising ways (WAPBL errors, missing PAM modules, broken login), the fix is almost always to add a keep-rule in sailor.vars or the service's own sailor.conf, not to patch mkimg.sh — see §20.

5.6 packages/ — Offline Binary Packages

Place pre-built .tgz packages here. mkimg.sh rsyncs them to the image root as /packages/. The pkgin common script detects /packages/ and installs them via pkg_add.

5.7 NETBSD_ONLY — Platform Marker

If this empty file exists, mkimg.sh refuses to build on non-NetBSD hosts:

This image must be built on NetBSD!
Use the image builder instead: make SERVICE=<name> build

6. Build Pipeline — Deep Dive

6.1 Image Creation (bmake SERVICE=foo build)

The build target in the Makefile orchestrates a two-stage process:

Stage 1: Builder microVM creation

bmake buildimg

1. SERVICE=build IMGTAG= base — calls mkimg.sh to create images/build-amd64.img
2. Extracts base + etc sets (plus partial comp:/usr/bin/strip) with MOUNTRO=y
3. Creates FFS (NetBSD) or ext2 (Linux) filesystem on the image
4. Installs the builder's own /etc/rc that waits for a second drive and executes build commands

Stage 2: Service build inside builder VM

bmake SERVICE=foo build

1. fetchall — download sets, packages, and kernel
2. Creates a blank disk image of IMGSIZE MB (via dd)
3. Writes ENVVARS to tmp/build-foo (lock/coordination file)
4. Launches the builder VM with startnb.sh:
   • -k kernels/netbsd-SMOL — PVH kernel
   • -i images/build-amd64.img — builder rootfs
   • -l images/foo-amd64.img — second drive (target image; -l also accepts a comma-separated list for multiple extra drives)
   • -w . — 9P share of project directory
   • -p ::22022-:22 — SSH access
   • -c $BUILDCPUS -m $BUILDMEM (defaults 2 cores / 1024 MB)
   • -x "-pidfile qemu-<service>.pid"
5. Builder VM's /etc/rc detects the second drive, sources tmp/build-foo, calls make base to invoke mkimg.sh to populate the target image
6. Builder removes tmp/build-foo when done (a final cat keeps the VM alive after)
7. Host polls the lock file, then kills builder QEMU via the pidfile
8. If MINIMIZE is set, waits for the image to be released (lsof), then resizes via qemu-img resize --shrink $(cat tmp/<img>.size) — if the image also uses WAPBL journaling, see §20.1
9. Writes signature to image and .sig file: smolsig:DD/MM/YYYY|UUID

6.2 mkimg.sh — Internal Flow

1. Source tmp/build-* for ENVVARS (SERVICE, ARCH, PKGVERS, etc.)
2. Source service/common/vars, funcs, choupi
3. Detect OS: NetBSD (a smolBSD builder image reports smolBSD — both accepted), Linux (ext2 path), FreeBSD (gpart/mdconfig path). OpenBSD, macOS and anything else exit 1. MINIMIZE or a NETBSD_ONLY marker also aborts on non-NetBSD hosts.
4. If FROMIMG is set, copy existing image (or dd onto the secondary disk); otherwise dd zero-filled image (host path only)
5. Partition and format:
   • Linux: sgdisk + losetup + mke2fs -O none (ext2, no journal)
   • FreeBSD: gpart + mdconfig + newfs (FFS)
   • NetBSD: gpt + vndconfig/dkctl wedge lookup + newfs (FFS). WAPBL log is only enabled when not MINIMIZE (noatime always; this is the guard behind §20.1)
6. Extract ADDPKGS packages into ${LOCALBASE} (e.g., /usr/pkg); exception: with sailor minimization requested, packages are cleanly reinstalled via pkgin from /tmp/usrpkg.tgz (backed up by the builder VM's /etc/rc)
7. If MINIMIZE + sailor.conf exists + /var/db/pkgin present: run sailor — cd ${BASEPATH}/sailor && ./sailor.sh build /service/<svc>/sailor.conf (seeded by service/common/sailor.vars, see §5.5)
8. Extract sets (tar xfp) — supports partial extraction (set:path); or copy a hand-made rootdir tar
9. Write /etc/fstab atomically (single full-content write): NAME=<svc>root / <fs> <opts> 1 1 — see §20.2 for a real corruption bug if this step is touched
10. Rsync service/<svc>/etc/ → mounted /etc/
11. Rsync service/common/ → mounted /etc/include/
12. Rsync service/<svc>/packages/ → mounted / (as /packages/)
13. Copy kernel if specified (-k, to /netbsd)
14. cd into mounted root; rescue symlink shims; run postinst/*.sh scripts sequentially (sorted by ls, sh $x, SVCIMG filter)
15. On non-NetBSD: backup MAKEDEV to etc/, patch unionfs out of dev/MAKEDEV (atomic mv)
16. Write PKGVERS to etc/pkgvers
17. If CURLSH set: curl -sSL | /bin/sh
18. If MINIMIZE: clean /var/db/pkgin
19. Create /var/qemufwcfg mount point
20. If BIOS boot: copy /usr/mdec/boot, create boot.cfg
21. Unmount; if MINIMIZE: du -s + resize_ffs -y -s + fsck_ffs -c4 -f -y, write new size in bytes to tmp/<img>.size
22. Detach loopback/vnd; if BIOS boot (NetBSD only): gpt biosboot -i 1 + installboot /usr/mdec/bootxx_ffsv1

Mount point selection:

• Host build (no secondary disk): mnt/ directory in project root
• Builder VM (secondary disk): /drive2 (from service/common/vars)

6.3 Builder VM (service/build/)

The builder VM is a special service that:

1. Sources basicrc and mount9p for networking and host sharing
2. Sets up SSL certificates for HTTPS fetching (tmpfs /etc/openssl + certctl rehash)
3. Sources tmp/build-* to get the target service's build variables
4. If MINIMIZE + sailor.conf: mounts tmpfs on /var/db and /usr/pkg, backs up /usr/pkg to /tmp/usrpkg.tgz (consumed by mkimg.sh's sailor path), and warns if the sailor/ clone is missing
5. Calls make <exported vars> base to invoke mkimg.sh for the target service (ADDPKGS is excluded from the make invocation)
6. Removes tmp/build-* when done (signals the host to kill the VM), then waits on cat

7. Boot / Runtime Pipeline — Deep Dive

7.1 startnb.sh — VM Launcher

Key flags:

Environment variables:

• QEMU_ACCEL — force a specific accelerator (kvm, hvf, nvmm, tcg)
• QEMU — override QEMU binary (default: qemu-system-<machine>)
• ARCH — override architecture detection

Architecture-specific QEMU invocation:

CPU overrides: QEMU_ACCEL=tcg forces qemu64 (x86) or max (aarch64) instead of host; on macOS Intel, cputype is always forced to qemu64.

Console detection:

• Checks kernel symbols for viocon_earlyinit via nm
• If found: uses VirtIO console (virtio-serial-device + virtconsole)
• If not: falls back to ISA serial console (-serial stdio or -serial pty)

Port forwarding format transformation:

# User input:   ::8080-:80
# Transformed:  hostfwd=tcp::8080-:80

The -p flag is processed by sed into QEMU hostfwd= syntax. The protocol prefix (tcp:, udp:) is optional and defaults to tcp. Multiple comma-separated pairs in one -p are all transformed.

PTY mode: When -P is passed:

1. QEMU starts with -daemonize and writes PTY path to qemu-<svc>.pty
2. startnb.sh waits for the file, extracts the PTY path, launches picocom -q -b 115200 (or the custom use_pty command)
3. On console exit, kills the QEMU process
4. This is the attach path used for services whose CMD runs login -f — the PTY console is what makes login's terminal handling behave correctly; see §20.3 for a related failure mode when the image has been minimized

RTC: Always -rtc base=utc,clock=host,driftfix=slew (UTC, host clock, slew driftfix) — added unconditionally.

QEMU 9.0/9.1 workaround: Adds -L bios -bios bios-microvm.bin to avoid stack smashing.

7.2 PVH Boot Flow

QEMU loads netbsd-SMOL kernel directly (no BIOS, no bootloader)
  → Kernel initializes VirtIO MMIO devices
  → Kernel mounts root filesystem (ld0 / dk0 / md0)
  → Kernel executes /etc/rc
    → . /etc/include/basicrc
      → Sets PATH, umask, HOME
      → Checks for md0 (ramdisk root), remounts rw if needed
      → mount -a (reads /etc/fstab)
      → Sources /etc/rc.pre (if exists)
      → Loads qemufwcfg variables (if mount_qemufwcfg available)
      → Creates /dev/MAKEDEV, mounts ptyfs, creates fd and ttyVI* devices
      → Configures vioif0: 10.0.2.15/24, gateway 10.0.2.2, DNS 10.0.2.3
      → Configures lo0 (IPv4 + IPv6)
      → Tunes TCP sendbuf/recvbuf
      → Sources /etc/include/mount9p (if vio9 device present)
      → Handles SSL_CERT_FILE (custom certs + certctl rehash)
      → Sources /etc/rc.local
      → If MOUNTRO: remount / read-only
    → . /etc/include/mount9p (if not already in basicrc)
    → Service-specific commands
    → CMD/ENTRYPOINT
    → . /etc/include/shutdown
      → sync, sync
      → If viocon: remount ro, echo 'JEMATA!' > /dev/ttyVI01
      → Else: umount -af
      → halt -lq

7.3 VM Control Socket

When -n N is used (N >= 1):

• Creates N VirtIO console sockets on the host (s-<uuid>-p<1..N>.sock in the CWD)
• /dev/ttyVI01 on the guest is the first socket (p1)
• startnb.sh spawns a background process that monitors the p1 socket via socat
• When guest writes JEMATA! to /dev/ttyVI01, the host kills QEMU
• Guest's shutdown script uses this for clean host-side teardown

8. Common Runtime Scripts — Reference

Snippets below reflect the current code at the version of this doc — they are reference, not specification. If you edit files in service/common/, keep §8 in sync.

8.1 basicrc (/etc/include/basicrc in VM)

export HOME=/
export PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/pkg/bin:/usr/pkg/sbin:/rescue
umask 022

# Handle ramdisk root (md0) — remount r/w
if [ "$(sysctl -n kern.root_device)" = "md0" ]; then
    mount -u -o rw /dev/md0a /
    sed -i'' 's,^[^ ]*,/dev/md0a,' /etc/fstab
fi

mount -a

# Optional pre-boot hook
[ -f /etc/rc.pre ] && . /etc/rc.pre

# QEMU fw_cfg variables (if mount_qemufwcfg binary exists)
if [ -f /sbin/mount_qemufwcfg ]; then
    dmesg | grep -q qemufwcfg && . /etc/include/qemufwcfg
fi

# Device nodes
[ ! -f "/dev/MAKEDEV" ] && cp -f /etc/MAKEDEV* /dev
if [ -f /sbin/mount_ptyfs ]; then
    mount -t ptyfs ptyfs /dev/pts
    cd /dev && sh MAKEDEV fd && cd -
    /etc/rc.d/ttys start
fi
chmod 666 /dev/null

# VirtIO console extra ports
dmesg | grep -q viocon && \
    (cd /dev && sh MAKEDEV ttyVI01 ttyVI02 && chmod 666 /dev/ttyVI*)

# Static IP (faster than DHCP)
if ifconfig vioif0 >/dev/null 2>&1; then
    route flush -inet6 >/dev/null 2>&1
    ifconfig vioif0 10.0.2.15/24
    route add default 10.0.2.2
    mount | grep read-only || echo "nameserver 10.0.2.3" > /etc/resolv.conf
fi

ifconfig lo0 127.0.0.1 up
ifconfig lo0 inet6 ::1 prefixlen 127 up

# TCP tuning
sysctl -w net.inet.tcp.sendbuf_max=16777216
sysctl -w net.inet.tcp.recvbuf_max=16777216
sysctl -w kern.sbmax=16777216

# 9P mount (if vio9 device present)
. /etc/include/mount9p

# Custom SSL certificates
if [ -n "$SSL_CERT_FILE" ]; then
    [ -d "$SSL_CERT_FILE" ] && cp -R "$SSL_CERT_FILE"/* /usr/share/certs/mozilla/server || \
        cp "$SSL_CERT_FILE" /usr/share/certs/mozilla/server
    certctl rehash 2>/dev/null
fi

# Optional post-boot hook
[ -f /etc/rc.local ] && . /etc/rc.local

# Read-only root if MOUNTRO env var set
[ -n "$MOUNTRO" ] && mount -u -o ro /

8.2 shutdown (/etc/include/shutdown)

sync; sync
dmesg | grep -q 'viocon0: adding port' && \
    (
        mount -u -o ro /
        sync; sync
        echo 'JEMATA!' > /dev/ttyVI01
    ) || \
umount -af
# no viocon(4) support
halt -lq

The JEMATA! signal tells the host-side socket monitor to kill QEMU cleanly.

8.3 mount9p (/etc/include/mount9p)

[ -z "$MOUNT9P" ] && MOUNT9P=/mnt

if ! mount | grep -q ${MOUNT9P} && dmesg | grep -q vio9; then
    [ -f /etc/MAKEDEV ] && cp /etc/MAKEDEV /dev
    cd /dev && sh MAKEDEV vio9p0
    cd -
    mount_9p -cu /dev/vio9p0 $MOUNT9P
fi

8.4 qemufwcfg (/etc/include/qemufwcfg)

QEMUFWCFG=/var/qemufwcfg
/sbin/mount_qemufwcfg $QEMUFWCFG

for file in ${QEMUFWCFG}/opt/org.smolbsd.var.*; do
    [ ! -f $file ] && continue
    VARNAME=${file##*.}
    eval "export $VARNAME=\$(cat \$file)"
done

8.5 funcs (service/common/funcs)

rsynclite()
{
    src=$1; dst=$2
    # Handle --exclude= flags
    # If src is a file: cp -f
    # If src is a directory: tar-based sync (faster than rsync for small trees)
}

8.6 vars (service/common/vars)

BASEPATH=/mnt       # Project root path (used by builder VM)
DRIVE2=/drive2      # Secondary disk mount point (used by builder VM)
CHOUPI=y            # Enable emoji output

8.7 choupi (service/common/choupi)

Sets emoji or ASCII fallback icons based on $CHOUPI:

• CHOUPI=y (default): emojis (➡️, ✅, ⚠️, ❌, …)
• CHOUPI=n: ASCII fallbacks (>, , !, X, …)

Also defines LIGHTGREEN, BOLD, NORMAL ANSI color codes.

8.8 pkgin (service/common/pkgin)

Bootstraps the pkgin package manager from scratch:

1. Detects NetBSD version → derives pkgsrc version
2. Determines architecture URL
3. Installs pkg_install, pkgin, pkg_tarup, rsync, curl via pkg_add
4. Configures repository URL
5. Installs packages from /packages/ directory if present
6. Installs $INSTALL_PACKAGES if set

9. OCI Registry (Push/Pull with oras)

Default registry: ghcr.io/netbsdfr/smolbsd Override with SMOLREPO environment variable.

# Push
./smoler.sh push myapp-amd64:latest
# → oras push ghcr.io/netbsdfr/smolbsd/myapp-amd64:latest \
#     --artifact-type application/vnd.smolbsd.image \
#     images/myapp-amd64.img:application/x-raw-disk-image

# Pull
./smoler.sh pull myapp-amd64:latest
# → oras pull ghcr.io/netbsdfr/smolbsd/myapp-amd64:latest
#   Downloads myapp-amd64.img to the CURRENT directory — move it to
#   images/ before building on top of or running it

# List images with signature verification
./smoler.sh images          # all images
./smoler.sh images ok       # only images with valid signatures

smoler/img.sh auto-installs oras binary to bin/oras if missing. Push/pull only work on Linux and macOS (other OSes are rejected). images additionally: reads the smolsig from the last 56 bytes of each image, verifies it against the .sig file (uuid only), and auto-recreates a missing .sig file for signed images (e.g. freshly downloaded ones).

Image naming: <service>-<arch>[:<tag>].img

• Tag defaults to latest
• Signature: smolsig:DD/MM/YYYY|UUID appended to image and stored in .sig file

10. GitHub Actions CI/CD

Two workflows in .github/workflows/:

main.yml — builder + rescue images

Triggers:

• Push to main (ignoring **.md, www/, app/, .github/workflows/smoler.yml, smolerfiles/*)
• Manual workflow_dispatch with inputs: img, arch, service, mountro, curlsh

Steps:

1. Checkout on ubuntu-latest in privileged debian:latest container
2. Install prerequisites: curl xz-utils make sudo git libarchive-tools rsync bmake e2fsprogs gdisk
3. Build for both amd64 and evbarm-aarch64:

   bmake SERVICE=<service> CURLSH=<curlsh> ARCH=$arch MOUNTRO=y <img|buildimg>
   bmake SERVICE=rescue ARCH=$arch base   # always build rescue
   

4. Compress all .img files with xz -T0 -9e + generate SHA256 sums
5. Upload to GitHub Release tag latest (pre-release) via softprops/action-gh-release@v2

smoler.yml — SMOLerfile service images

Triggers on push to smolerfiles/* (or manual dispatch with a list of files). Builds a fixed set of services (crush, clawd, bsdshell, nbakery, tiny, clawlite, ttyd) with smoler.sh build on plain runners using QEMU_ACCEL=tcg (no KVM), after bmake fetchimg for amd64 and evbarm-aarch64; publishes to GitHub Packages (packages: write). New SMOLerfiles you want CI-tested must be added to the DFILES regex in this workflow.

Note: the CI runner is Linux-only (ext2 builder path), so any fix that's specific to the NetBSD FFS builder path (WAPBL, resize_ffs, sailor) will not be exercised by CI — those must be tested manually on a NetBSD host. See §20.

11. BIOS Boot & Bare Metal

When standard PVH boot is unavailable (Bhyve, bare metal, other VMMs):

# Build with BIOS boot
./smoler.sh build -y -t USB BIOSBOOT=y BIOSCONSOLE=pc smolerfiles/Dockerfile.bsdshell

# Bare metal: dd to USB drive
sudo dd if=images/bsdshell-amd64:USB.img of=/dev/sde bs=1M

# Bhyve/other VMMs: also SMOLIFY the kernel
./smoler.sh build -y -t freebsd BIOSBOOT=y SMOLIFY=y smolerfiles/Dockerfile.bsdshell

BIOS boot internals (in mkimg.sh):

• Copies /usr/mdec/boot to image
• Creates /boot.cfg with timeout=0 and consdev=${BIOSCONSOLE}
• After umount: gpt biosboot -i 1 ${imgdev}, installboot /dev/r${mountdev} /usr/mdec/bootxx_ffsv1

In kernfetch (Makefile):

• If BIOSBOOT=y, downloads netbsd-GENERIC kernel
• Copies as kernels/netbsd-GENERIC.SMOL
• If SMOLIFY=y and confkerndev exists: runs confkerndev to keep only essential drivers: mainbus cpu acpicpu ioapic pci isa pcdisplay wsdisplay com virtio ld vioif qemufwcfg

confkerndev — Kernel Driver Slimming

A C tool that modifies the kernel ELF binary to disable device drivers without recompilation:

# List all drivers
./confkern

…(truncated)