Packaging Static Installer

Building a static Netdata binary

The static build produces a self-extracting installer (netdata-<arch>-latest.gz.run) for a compatible Linux system matching the selected architecture, including supported 32-bit ARM targets. It installs under /opt/netdata without requiring a native build toolchain on the target.

Loading this skill does not authorize image pulls/removal, submodule changes, privileged host registration, builds, extraction or target installation. An authorized build may involve those build prerequisites: inspect the concrete requirements and apply existing authorization. Preserve unrelated submodule work and existing artifacts before any operation that would replace them. Building an artifact alone does not authorize deploying it.

Timing, size and slowdown figures below are historical observations, not build or compatibility guarantees.

TL;DR — x86_64 native

# 1. Pre-flight for an authorized build (inspect first; honor pinned reproduction inputs)
# Initialize only missing required submodules using Gotcha #1 below.
# For a normal build, refresh the image; retain the selected image for a pinned reproduction.
docker pull netdata/static-builder:v1     # see Gotcha #2

# 2. Build (~22-25 min cold, ~10-15 min on cache hit, on a 24-core host)
./packaging/makeself/build-static.sh x86_64

# 3. Output
ls -la artifacts/
# artifacts/netdata-x86_64-latest.gz.run        historically ~190 MB; matching compatible x86_64 Linux target
# artifacts/netdata-x86_64-vX.Y.Z-N-nightly.gz.run
# artifacts/netdata-latest.gz.run               (x86_64 only — alias of the above)
# artifacts/netdata-vX.Y.Z-N-nightly.gz.run     (x86_64 only — alias)

For debug builds: ./packaging/makeself/build-static.sh x86_64 debug.

What you actually run

The orchestrator is packaging/makeself/build-static.sh, which:

1. Translates the architecture name (x86_64, aarch64, armv6l, armv7l) to a docker --platform value via packaging/makeself/uname2platform.sh.
2. Sets per-arch tuning flags (packaging/makeself/build-static.sh:27-56):
   • x86_64 → -march=x86-64 baseline (distinct from the Nehalem-v2 QEMU CPU choice), GOAMD64=v1.
   • aarch64 → -march=armv8-a, Cortex-A53, GOARM64=v8.0.
   • armv7l → -march=armv7-a, Cortex-A7, GOARM=7.
   • armv6l → -march=armv6zk -mtune=arm1176jzf-s, ARM1176, GOARM=6.
3. Registers binfmt/QEMU on the container host using a privileged container when cross-arch emulation is needed. Existing registration or SKIP_EMULATION skips that operation.
4. Pulls netdata/static-builder:v1 if missing locally; removes a mismatched-platform image first.
5. Bind-mounts $(pwd) into the container at /netdata and runs /netdata/packaging/makeself/build.sh inside it.

The container then runs packaging/makeself/run-all-jobs.sh, which executes packaging/makeself/jobs/*.sh in lexical order:

Source files: packaging/makeself/build-static.sh, packaging/makeself/build.sh, packaging/makeself/run-all-jobs.sh, packaging/makeself/functions.sh, packaging/makeself/jobs/.

Pre-flight (DO NOT SKIP)

For an authorized build, establish these prerequisites before spending build time. The launcher bind-mounts the checkout writable and replaces generated artifacts; preserve existing results that must survive the build.

Gotcha #1: submodules must be initialized

The build configures with CMake against vendored sources at:

• src/aclk/aclk-schemas/
• src/collectors/debugfs.plugin/libsensors/vendored/

If either is empty, CMake aborts after ~2 min with:

Cannot find source file: vendored/lib/access.c
No SOURCES given to target: vendored_libsensors
ABORTED  Failed to configure Netdata sources.

A plain fresh clone or linked worktree may have uninitialized submodules. Inspect git submodule status and local submodule changes first. Initialize missing required sources for the authorized build; do not reset modified or divergent submodules merely to make the status clean. For each required path listed above whose status begins with -, set missing_required_submodule to that exact path in the same shell invocation as the command below. Repeat that assignment and command once per selected path; do not assume shell variables persist across separate tool calls:

git submodule update --init -- "${missing_required_submodule:?set one uninitialized required submodule path}"

The path argument confines initialization to the selected missing module. Do not use an unscoped or recursive update: that can move already initialized modules to recorded commits, including unrelated or deliberately divergent modules.

Verify with git submodule status: - means uninitialized, + differs from the recorded commit, and U is conflicted. A leading space confirms the recorded commit, not the absence of local edits. Investigate differences before proceeding.

Gotcha #2: refresh the cached docker image

The launcher pulls the target-platform image only when it is missing; it removes a cached image with a mismatched platform first. A previously observed stale image failed checksum verification with:

sha256sum: unrecognized option: c
SHA256 verification of tar file libnetfilter_acct-1.0.3.tar.bz2 failed (rc=1)
expected: <hash>, got <same-hash>

functions.sh uses sha256sum --c --status; an implementation lacking that option can fail despite matching bytes. This symptom is evidence to inspect the image/tool, not proof of a specific Alpine version. For a normal build session, refresh the intended platform's image and record its identity; a deliberately pinned reproduction should retain its selected image. The quick-start pull above targets native x86_64; use --platform from uname2platform.sh for other targets. A pull can replace the cached tag; current registry contents are not established by this skill.

Output artifacts

Job 99-copy-archives.sh writes to artifacts/ (gitignored, host-side, owned by your user):

artifacts/
├── netdata-x86_64-latest.gz.run            # copied alias of the versioned archive
├── netdata-x86_64-v2.10.0-171-nightly.gz.run
├── netdata-latest.gz.run                   # x86_64 only — generic alias
├── netdata-v2.10.0-171-nightly.gz.run      # x86_64 only — generic alias
└── cache/                                  # build cache, see below

For non-x86_64 builds, only the two netdata-<arch>-* files are produced (packaging/makeself/jobs/99-copy-archives.sh:25-30).

Verify a build:

ls -la artifacts/
sha256sum artifacts/netdata-x86_64-latest.gz.run

# What's inside (read-only inspection, does not run the installer)
sh artifacts/netdata-x86_64-latest.gz.run --info
sh artifacts/netdata-x86_64-latest.gz.run --list | head

Each .gz.run is a makeself archive: a shell prefix that extracts the gzipped tar embedded after it. Run it as root on the target to install.

Build cache

artifacts/cache/<arch>/ holds the compiled third-party deps (openssl, curl, bash, libunwind, libnetfilter_acct, ioping) keyed by their pinned source versions in packaging/makeself/bundled-packages.version. The fetch logic is in packaging/makeself/functions.sh (cache_path(), fetch(), fetch_git(), store_cache()).

Implications:

• First build: ~22-25 min on a 24-core host (most time = third-party compile + Rust crate compile + LTO link of the C plugins).
• Cache hit: ~10-15 min (skips the third-party deps; only the netdata sources rebuild).
• The cache survives git checkout and git clean -fd (it's under the gitignored artifacts/).
• Bumping a version in bundled-packages.version invalidates that one entry — the rest still reuse.
• Cache entries are version-derived directories under <arch>/<package>/; image contents and compiler flags are not part of that key. For a cold rebuild, preserve the exact cache being invalidated outside the active cache path, or build in an isolated checkout. Do not delete all of artifacts/: it also holds installers and other architectures. Any deletion still needs the existing task authorization.

Cross-architecture builds (aarch64, armv7l, armv6l)

./packaging/makeself/build-static.sh aarch64
./packaging/makeself/build-static.sh armv7l
./packaging/makeself/build-static.sh armv6l

The script auto-installs QEMU binfmt handlers via tonistiigi/binfmt:master if not already registered (packaging/makeself/build-static.sh:60-62). Cross-arch builds:

• Run all C/Rust/Go compilation under QEMU emulation — expect 4-8× slowdown vs native.
• Are CPU-bound, not network-bound — the source download is one-time.
• Can fail in ways native builds do not (e.g. Rust LTO under QEMU has historically OOMed; libbpf BPF skeleton generation has hit qemu syscall edge cases). When you see a failure that isn't on x86_64 native, suspect QEMU first.

SKIP_EMULATION=1 is set automatically when the host already has a binfmt_misc entry for the target arch (e.g. on a CI runner with persistent qemu).

Debug builds

./packaging/makeself/build-static.sh x86_64 debug

Sets NETDATA_BUILD_WITH_DEBUG=1 (packaging/makeself/build.sh:9-22), which selects reduced C optimization (-O1 -ggdb) and internal checks in the Netdata build job. Historically the archive was about twice the size, with slower runtime useful for valgrind/gdb. The README.md in packaging/makeself/ documents valgrind invocation.

Common failures

The build script exits with Build failed. on any job failure (packaging/makeself/build.sh:44-52). For an interactive TTY launch, DEBUG_BUILD_INFRA=1 is forwarded and opens a bash shell on failure; the non-TTY branch does not forward it. Use an interactive launch when that diagnostic shell is needed.

Watching a long-running build

Prefer the execution tool's tracked session for a long build. If using a shell background job, give it a fresh task directory and capture its PID immediately:

BUILD_RUN="$(mktemp -d "${TMPDIR:-/tmp}/netdata-build.XXXXXX")" || exit $?
LOG="$BUILD_RUN/build.log"
nohup ./packaging/makeself/build-static.sh x86_64 > "$LOG" 2>&1 &
BUILD_PID=$!
printf '%s\n' "$BUILD_PID" > "$BUILD_RUN/build.pid"

# Progress
grep -E '^ --- running' "$LOG"          # job-level milestones
tail -f "$LOG"                          # streaming output
docker stats --no-stream                # CPU/RAM of the running container

Before stopping it, verify the recorded PID still belongs to this task; stopping the launcher does not prove its container stopped. Inspect the task container identity separately and never terminate by a broad process name.

Indicators the build is alive (output buffering can stall the log for minutes during heavy compile):

• ps -eo pid,pcpu,comm --sort=-pcpu | head shows rustc, cc1, lto1-ltrans near 50-70% each.
• docker stats shows the static-builder container at hundreds of % CPU.
• The container's working set in docker stats keeps changing.

Deploying to a target

Use this only for the requested target installation, with a matching architecture and the requested update policy.

# Copy
scp artifacts/netdata-x86_64-latest.gz.run target-host:/tmp/

# Install (on the target, as root)
ssh target-host
sudo sh /tmp/netdata-x86_64-latest.gz.run -- --auto-update
# installer flags after the bare `--`; common: --dont-start-it, --disable-telemetry,
# --claim-token <T> --claim-rooms <R>, --no-updates

The installer always installs into /opt/netdata (hard-coded; --target would change it but the in-archive paths assume /opt/netdata, do not override).

Read-only archive metadata/listing:

sh artifacts/netdata-x86_64-latest.gz.run --info     # makeself metadata
sh artifacts/netdata-x86_64-latest.gz.run --list     # full file manifest

For requested extraction, use a fresh directory. --noexec skips the installer but still writes files and does not bypass the archive's root requirement. --target here controls extraction, not a supported alternate install prefix.

EXTRACT_DIR="$(mktemp -d "${TMPDIR:-/tmp}/netdata-extract.XXXXXX")" || exit $?
sudo sh artifacts/netdata-x86_64-latest.gz.run --target "$EXTRACT_DIR" --noexec --keep

Keep the extracted files for inspection; do not reuse an existing destination or remove unrelated files.

How to extend this skill

Capture timing and authorization follow AGENTS.md#knowledge-capture. Authorized recipes for build failures, architecture-specific quirks, and reusable workflows belong in how-tos/, with an entry in ./how-tos/INDEX.md. Keep SKILL.md focused on the workflow and route detailed recipes through the catalog.

Source-of-truth pointers

• packaging/makeself/README.md — high-level user-facing doc (architectures, valgrind notes).
• packaging/makeself/build-static.sh — host-side launcher, arch matrix, docker invocation.
• packaging/makeself/build.sh — in-container entry; debug-flag parsing.
• packaging/makeself/run-all-jobs.sh — job runner.
• packaging/makeself/functions.sh — cache_path(), fetch(), fetch_git(), store_cache(), progress(), run() helpers.
• packaging/makeself/jobs/*.sh — the ordered build steps.
• packaging/makeself/bundled-packages.version — pinned versions of openssl, curl, bash, libunwind, libnetfilter_acct, ioping.
• packaging/makeself/install-alpine-packages.sh — the package list the static-builder docker image is built from (used when refreshing the image, not on every build).
• packaging/makeself/uname2platform.sh — arch → docker --platform translation.
• packaging/makeself/makeself.sh, makeself-header.sh — vendored makeself archive builder.