C Incremental Build Converter

C/C++ Incremental Build Converter Skill

This skill converts C/C++ project build.sh and test.sh scripts to support incremental builds after docker commit.

CRITICAL CONSTRAINTS

1. DO NOT Modify oss-fuzz Infrastructure Code

NEVER modify these files:

• oss-fuzz/infra/helper.py
• oss-fuzz/infra/base-images/*
• Any other oss-fuzz infrastructure code

ONLY modify:

• oss-fuzz/projects/{project}/build.sh
• oss-fuzz/projects/{project}/test.sh
• oss-fuzz/projects/{project}/Dockerfile (if absolutely necessary)

2. If oss-crs Test Framework Has Issues

If the incremental build test framework (incremental_build_checker.py) has bugs or limitations:

1. Report the issue to the user with clear explanation
2. Suggest a fix for the framework code
3. DO NOT implement the fix yourself - let the user decide

Example report format:

## oss-crs Framework Issue Detected

**File:** bug_fixing/src/oss_patch/inc_build_checker/incremental_build_checker.py
**Issue:** The source copy command destroys build artifacts
**Current behavior:** `rm -rf $SRC/nginx && cp -r ...`
**Expected behavior:** Should preserve objs/ directory
**Suggested fix:** Use rsync with --exclude objs instead of rm+cp

Key Architecture: Separate $SRC for build.sh and test.sh

IMPORTANT: $SRC is different between build.sh and test.sh.

• build.sh runs in /built-src → $SRC = /built-src
• test.sh runs in /test-src → $SRC = /test-src

These are completely separate directories. This separation means:

1. NO artifact cleanup needed - Don't delete .o, .lo, .a, .la, .libs
2. NO out-of-tree build needed - In-source build is fine since $SRC is separate
3. NO ASAN conflict - test.sh builds in its own clean copy

ALWAYS Use $SRC, NEVER Hardcode Paths

CRITICAL: Always use $SRC variable, never hardcode /src, /built-src, or /test-src.

# BAD - Hardcoded path breaks when $SRC changes
cd /src/project
cp -r /src/oniguruma modules/

# GOOD - Uses $SRC variable
cd $SRC/project
cp -r $SRC/oniguruma modules/

AVOID $WORK - Use $SRC Subdirectories for All Build Artifacts

CRITICAL: Avoid using $WORK for build-related directories. Use $SRC subdirectories instead.

$WORK is shared between build.sh and test.sh, which causes problems:

1. ASAN-instrumented libraries from build.sh conflict with test.sh
2. CMake paths recorded in $WORK become invalid when $SRC changes
3. Build artifacts from different environments can corrupt each other

Always use $SRC subdirectories for:

• Build directories (out-of-source builds)
• Install prefixes (for cmake/configure --prefix)
• Dependency installations

# BAD - Uses $WORK which is shared
BUILD_DIR="$WORK/build"
cmake "$SRC/project" -DCMAKE_INSTALL_PREFIX=$WORK

# GOOD - Uses $SRC subdirectories
BUILD_DIR="$SRC/project_build"
INSTALL_DIR="$SRC/project_install"
mkdir -p "$BUILD_DIR" "$INSTALL_DIR"
cd "$BUILD_DIR"
cmake "$SRC/project" -DCMAKE_INSTALL_PREFIX="$INSTALL_DIR"

In test.sh, also use $SRC subdirectories:

# BAD in test.sh - Uses shared $WORK with ASAN artifacts
./configure --prefix="$WORK"
# Error: undefined reference to __asan_*

# GOOD in test.sh - Uses separate prefix under $SRC
TEST_PREFIX="$SRC/test_deps"
mkdir -p "$TEST_PREFIX/lib" "$TEST_PREFIX/include"

if [ ! -f "$TEST_PREFIX/lib/libz.a" ]; then
    pushd "$SRC/zlib"
    ./configure --static --prefix="$TEST_PREFIX"
    make -j$(nproc) CFLAGS="-fPIC"  # No ASAN flags
    make install
    popd
fi

test.sh Does NOT Need Sanitizers - Disable Them Explicitly

IMPORTANT: test.sh runs unit tests, NOT fuzzing. Sanitizers are NOT needed and cause conflicts.

During incremental build testing, build.sh compiles with ASAN/sanitizer flags. When test.sh runs in the SAME directory, the object files from build.sh are ASAN-instrumented. If test.sh tries to link against these objects without ASAN runtime, you get undefined reference errors.

build.sh and test.sh Artifacts Should NOT Be Shared

CRITICAL: Build artifacts from build.sh should NOT be reused in test.sh.

During incremental build testing:

1. build.sh runs with ASAN flags → creates ASAN-instrumented .o files
2. test.sh runs in the SAME $SRC directory
3. If test.sh tries to use existing .o files, linking fails

The artifacts are incompatible because:

• build.sh: Compiles with -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link
• test.sh: Needs to link without sanitizer runtime

DO NOT use || true to ignore link failures - this hides the real problem. Fix the root cause by ensuring clean separation of build artifacts.

Critical Requirements

1. Scripts MUST Work on BOTH First Run AND Repeated Runs

NEVER assume a script will only run once. Scripts must be idempotent.

# BAD - Runs configure every time
autoreconf -f -i
./configure

# GOOD - Only configure on first run
if [ ! -f Makefile ]; then
    autoreconf -f -i
    ./configure
fi
make -j$(nproc)

2. DO NOT Delete Build Artifacts

Since $SRC is separate, there's no need to clean artifacts. Deleting them defeats incremental build.

# BAD - Destroys incremental build benefits
find . -name "*.o" -delete
find . -name "*.a" -delete
rm -rf .libs

# GOOD - Just build, artifacts from previous run will speed things up
if [ ! -f Makefile ]; then
    ./configure
fi
make -j$(nproc)

3. Conditional Configure/Autoreconf

Only run configure-related commands on first execution.

# Pattern: Check for Makefile existence
if [ ! -f Makefile ]; then
    autoreconf -fi       # or ./bootstrap
    ./configure --options
fi
make -j$(nproc)
make check

4. Use cp Instead of mv

mv removes the source file, causing failures on repeated runs.

# BAD - Fails on second run
mv scripts/config.temp scripts/config

# GOOD - Preserves source file
[ -f scripts/config.temp ] && cp scripts/config.temp scripts/config || true

5. Use Guards for Dependencies

# Build dependency only if not already built
if [ ! -f "$WORK/lib/libz.a" ]; then
    pushd "$SRC/zlib"
    ./configure --static --prefix="$WORK"
    make -j$(nproc)
    make install
    popd
fi

6. Handle Failing Tests

Remove failing test files directly instead of using TESTS variable (which can propagate to submodules).

# BAD - TESTS variable propagates to submodules
make check TESTS="tests/mantest tests/jqtest"

# GOOD - Remove failing test files directly
rm -f tests/shtest
rm -f tests/multiple.*
sed -i '/multiple/d' tests/Makefile.am
make check

7. Copy Dependencies If Not Present (Conditional, Not Delete-and-Copy)

# BAD - Deletes and recopies every time, breaks incremental build
rm -rf modules/oniguruma
rsync -a $SRC/oniguruma modules/

# GOOD - Only copy if not present
if [ ! -d modules/oniguruma ]; then
    mkdir -p modules
    cp -r $SRC/oniguruma modules/
fi

8. Conditional Dict/Corpus File Copy

Files like dictionaries and seed corpus may not exist in all builds.

# BAD - Fails if file doesn't exist
cp $SRC/libtiff/contrib/oss-fuzz/tiff.dict $OUT/

# GOOD - Conditional copy with fallback
[ -f $SRC/libtiff/contrib/oss-fuzz/tiff.dict ] && cp $SRC/libtiff/contrib/oss-fuzz/tiff.dict $OUT/ || true
find . -name "*.tif" | xargs zip $OUT/seed_corpus.zip || true

9. Use make -k check || true for Potentially Failing Tests

Some tests may fail in fuzzing environment. Use -k to continue and || true to not fail the script.

# BAD - Stops on first test failure
make check

# GOOD - Continue through failures
make -k check || true

echo "Tests completed"

10. Remove Tests from Makefile.am BEFORE configure

When removing failing tests, modify Makefile.am BEFORE running autoreconf/configure.

# BAD - Removes test file but Makefile still references it
rm -f tests/shtest
autoreconf -fi
./configure
make check  # Error: No rule to make target 'tests/shtest'

# GOOD - Remove from Makefile.am before configure
sed -i 's| tests/shtest||g' Makefile.am
autoreconf -fi
./configure
make check  # Works

11. Rewrite Instead of Sourcing Internal build.sh

If the project's source contains its own build.sh (e.g., contrib/oss-fuzz/build.sh), it may use mv or other non-idempotent operations. Rewrite the logic in the project's build.sh instead.

# BAD - Sources internal build.sh that uses mv
source $SRC/project/contrib/oss-fuzz/build.sh

# GOOD - Rewrite the logic with cp instead of mv
if [ ! -f "$WORK/lib/libjbig.a" ]; then
    pushd "$SRC/jbigkit"
    make lib
    # Use cp instead of mv for incremental build support
    cp "$SRC"/jbigkit/libjbig/*.a "$WORK/lib/"
    cp "$SRC"/jbigkit/libjbig/*.h "$WORK/include/"
    popd
fi

12. Guard sed -i Insertions with Pattern Checks

When using sed -i to insert content, check BOTH the original pattern exists AND the new content doesn't exist yet.

# BAD - Only checks if new content exists (can't distinguish original vs added)
if ! grep -q '%destructor' file.y; then
    sed -i '/^%%$/i%destructor { free ($$); } ID' file.y
fi

# GOOD - Check original pattern exists AND new content not yet added
if grep -q '^%%$' file.y && ! grep -q '%destructor' file.y; then
    sed -i '/^%%$/i%destructor { free ($$); } ID' file.y
fi

13. Guard Append Operations (>>)

Don't blindly use > (overwrite) or >> (append). Check if content already exists.

# BAD - Overwrites every time (inefficient, may lose state)
sed 's/main/old_main/' file.c > header.h

# BAD - Appends every time (file grows on each run)
sed 's/main/old_main/' file.c >> header.h

# GOOD - Only append if content not already present
if ! grep -q 'old_main' header.h 2>/dev/null; then
    sed 's/main/old_main/' file.c >> header.h
fi

14. CMake/Ninja Build Directory Must Use $SRC-Relative Path

When $SRC changes between build.sh and test.sh, CMake/Ninja will fail with "manifest dirty" errors if build directory is in $WORK.

# BAD - $WORK is shared, CMake source path mismatch causes ninja errors
BUILD_DIR="$WORK/build"
cd "$BUILD_DIR"
cmake $SRC/project  # Source path recorded as /built-src/project
# On test.sh: ninja fails because $SRC is now /test-src/project

# GOOD - Build directory relative to $SRC, each environment independent
BUILD_DIR="$SRC/project_build"
mkdir -p "$BUILD_DIR"
cd "$BUILD_DIR"
if [ ! -f build.ninja ]; then
    cmake -GNinja $SRC/project
fi
ninja

15. Disable ccache for Bazel Builds

Bazel has its own caching mechanism. ccache interferes with Bazel's compiler wrappers.

# BAD - ccache causes "invalid option" errors with Bazel
# Error: /usr/local/bin/ccache: invalid option -- 'U'
bazel build //...

# GOOD - Remove ccache from PATH before Bazel builds
export PATH=$(echo "$PATH" | sed 's|/ccache/bin:||g; s|:/ccache/bin||g')
unset CC CXX
export CC=clang
export CXX=clang++
bazel build //...

Standard test.sh Template

IMPORTANT: Execution Environment

• $SRC is already set (no need for : "${SRC:=/src}")
• Working directory is already $SRC/{project} (no need for cd $SRC/{project})

#!/bin/bash
set -ex

# Build (only configure on first run)
if [ ! -f Makefile ]; then
    autoreconf -fi  # or ./bootstrap, or skip if not autotools
    ./configure --options
fi
make -j$(nproc)

# Run tests (use -k to continue on failures, || true to not fail script)
make -k check || true

echo "Tests completed"

Examples by Build System

Autotools Project

#!/bin/bash
set -ex

if [ ! -f Makefile ]; then
    autoreconf -fi
    ./configure --enable-static
fi
make -j$(nproc)
make -k check || true

CMake Project

#!/bin/bash
set -ex

if [ ! -f Makefile ]; then
    cmake . -DCMAKE_INSTALL_PREFIX=$WORK -DBUILD_SHARED_LIBS=off
fi
make -j$(nproc)
make test

Project with Dependencies

#!/bin/bash
set -ex

# Build zlib (only on first run)
if [ ! -f "$WORK/lib/libz.a" ]; then
    pushd "$SRC/zlib"
    ./configure --static --prefix="$WORK"
    make -j$(nproc)
    make install
    popd
fi

# Build main project
if [ ! -f Makefile ]; then
    cmake . -DCMAKE_INSTALL_PREFIX=$WORK
fi
make -j$(nproc)
make test

Project with Submodules (e.g., jq with oniguruma)

#!/bin/bash
set -ex

# Copy submodule if not present (conditional, not delete-and-copy)
if [ ! -d modules/oniguruma ]; then
    mkdir -p modules
    cp -r $SRC/oniguruma modules/
fi

# Remove failing tests from Makefile.am BEFORE configure
sed -i 's| tests/shtest||g' Makefile.am

if [ ! -f Makefile ]; then
    autoreconf -fi
    ./configure --with-oniguruma=builtin
fi
make -j$(nproc)

# Run tests (some may fail in fuzzing environment)
make -k check || true

Common Errors and Fixes

Testing Incremental Builds

CRITICAL: Use the test-inc-build subagent via Task tool to prevent context limit errors.

The subagent runs tests, analyzes logs internally, and returns only essential summary information.

Single Project Test

Task tool:
  subagent_type: "test-inc-build"
  description: "Test incremental build"
  prompt: "Run incremental build test for {project_name} with oss-fuzz path ../oss-fuzz"

AIXCC Projects: Use Full Path Prefix

IMPORTANT: For AIXCC projects, use the aixcc/c/ or aixcc/jvm/ prefix in the project name.

Examples:

• aixcc/c/afc-sqlite3-delta-01
• aixcc/c/atlanta-libjpeg-full-01
• aixcc/jvm/afc-jenkins-delta-01

Parallel Testing Multiple Projects

Use Task tool with run_in_background: true in a single message:

Task tool (run_in_background: true):
  subagent_type: "test-inc-build"
  prompt: "Run incremental build test for aixcc/c/afc-libexif-delta-01 with oss-fuzz path ../oss-fuzz"

Task tool (run_in_background: true):
  subagent_type: "test-inc-build"
  prompt: "Run incremental build test for aixcc/c/afc-libexif-delta-02 with oss-fuzz path ../oss-fuzz"

Then use TaskOutput to retrieve results from each agent.

What the Test Does

1. Runs build.sh (first run)
2. Runs docker commit to save state
3. Runs build.sh again (incremental run)
4. Compares build times and reports reduction percentage

Subagent Return Format

On Success:

## Result: SUCCESS
**Project:** {project_name}
**Build Time Reduction:** {percentage}%
**Summary:** Incremental build working correctly.

On Failure:

## Result: FAILED
**Project:** {project_name}
**Error Type:** {category}
**Error Message:** {exact error, max 10 lines}
**Suggested Fix:** {brief fix}

Success criteria:

• Both runs complete without errors
• Build time reduction is reported (higher % = better incremental build support)

Checklist

• Uses $SRC variable, NOT hardcoded paths like /src or /built-src
• Uses if [ ! -f Makefile ] guard around configure/autoreconf
• Does NOT delete build artifacts (.o, .lo, .a, .la, .libs)
• Uses cp instead of mv where applicable
• Failing tests removed from Makefile.am BEFORE configure (not just file deletion)
• Dependencies have existence guards
• Directory copies are conditional (if [ ! -d ]), not delete-and-copy
• Uses $SRC subdirectories instead of $WORK for build dirs and install prefixes
• test.sh uses separate prefix (e.g., $SRC/test_deps) if dependencies needed
• Dict/corpus copies use conditional: [ -f file ] && cp ... || true
• Tests use make -k check || true if some may fail
• Internal build.sh rewritten if it uses mv or non-idempotent operations
• sed -i insertions guarded with pattern + content check
• Append operations (>>) guarded with content existence check
• CMake/Ninja build directories use $SRC-relative paths (not $WORK)
• Bazel builds have ccache removed from PATH
• Works on first run AND repeated runs