Kernel Testing

Kernel testing

Contract

Inputs

1. Code under test (required): a kernel function, driver helper, syscall surface, or a patch.
2. Kernel tree and build (required): the checkout the tests run against; mainline 7.2 or LTS 6.18 assumed.
3. Coverage or fuzzing target (optional): syzkaller VM budget, LTP subset.

Procedure

1. Choose the layer before writing anything.

   Pure kernel function logic            -> KUnit
   Userspace-visible behavior (syscall)  -> kselftest
   Crash and security hunting            -> syzkaller + KASAN
   Regression across distros             -> LTP
   Upstream patch CI                     -> build, boot, kselftest ladder
   

   Done when: the surface under test names its layer.

2. Write the KUnit suite for kernel-context logic.

   #include <kunit/test.h>
   
   static void example_test(struct kunit *test)
   {
       KUNIT_EXPECT_EQ(test, 1 + 1, 2);
   }
   
   static struct kunit_case cases[] = {
       KUNIT_CASE(example_test),
       {}
   };
   static struct kunit_suite suite = {
       .name = "example",
       .test_cases = cases,
   };
   kunit_test_suite(suite);
   MODULE_LICENSE("GPL");
   

   Allocate test memory with kunit_kmalloc so the test core frees it. Build with obj-$(CONFIG_KUNIT) += test_example.o and run:

   ./tools/testing/kunit/kunit.py run
   ./tools/testing/kunit/kunit.py run --filter=example_test
   ./tools/testing/kunit/kunit.py run --arch arm64 --cross_compile aarch64-linux-gnu-
   

   KUnit executes in kernel context, under UML by default or on a real target with --arch. Done when: kunit.py run reports the suite green.

3. Add the kselftest case for userspace-visible behavior. A new test lives in tools/testing/selftests/<name>/ with a Makefile, the sources, and a config file naming required Kconfig symbols.

   # tools/testing/selftests/mytest/Makefile
   CFLAGS += -Wall
   TEST_GEN_FILES := mytest
   TEST_PROGS := mytest
   include ../lib.mk
   

   cd tools/testing/selftests && make -j$(nproc)   # build
   make run_tests                                  # run all
   ./memfd/memfd_test                              # run one
   

   A SKIP result means a kernel feature is missing, not that the test passed. Done when: the test builds, runs, and skips only for the documented reason.

4. Fuzz the syscall surface with syzkaller. The kernel needs CONFIG_KCOV, CONFIG_DEBUG_FS, and ideally CONFIG_KASAN. Build syzkaller from source, point a manager config at the kernel build and image, then run.

   {
     "target": "linux/amd64",
     "http": "127.0.0.1:56741",
     "workdir": "/tmp/syzkaller",
     "kernel_obj": "/path/to/kernel/build",
     "syzkaller": "/path/to/syzkaller",
     "procs": 8,
     "type": "qemu",
     "vm": {"count": 4, "kernel": "/path/to/bzImage", "cpu": 2, "mem": 2048}
   }
   

   ./bin/syz-manager -config manager.cfg
   ./bin/syz-repro -config manager.cfg crash-report.txt   # minimize a crash
   

   Done when: the manager runs VMs and every found crash has a minimized reproducer.

5. Read coverage through kcov. CONFIG_KCOV exposes a debugfs character device that programs open, size with KCOV_INIT_TRACE, map, and enable with KCOV_ENABLE; syzkaller drives it automatically for every executed program. Zero coverage means the kernel was built without CONFIG_KCOV. Done when: coverage data flows for at least one program.

6. Run LTP for syscall regression across environments.

   git clone https://github.com/linux-test-project/ltp && cd ltp
   make autotools && ./configure && make -j$(nproc) && make install
   /opt/ltp/runltp -f syscalls          # full syscall set
   /opt/ltp/testcases/bin/pipe01        # one test, run directly
   

   LTP covers syscalls, filesystems, network, IPC, controllers, and security. Mass failure usually means the environment, not the kernel: run as root and check the documented prerequisites. Done when: the selected set reports a clean pass count.

7. Shape CI on the build, boot, test ladder. A kernel patch pipeline is: build with a chosen config or allmodconfig, boot under QEMU, kselftest, then an LTP subset. KernelCI runs this shape against many boards and defconfigs upstream; a private CI mirrors it on the configs the patch touches. Done when: the pipeline runs the ladder on every patch.

8. Start from a debug-friendly config.

   CONFIG_KUNIT=y
   CONFIG_KCOV=y
   CONFIG_KASAN=y
   CONFIG_DEBUG_INFO_DWARF5=y
   CONFIG_FRAME_POINTER=y
   CONFIG_FTRACE=y
   

   make kvm_guest.config        # reasonable test base on x86-64
   scripts/config -e KUNIT -e KCOV -e KASAN
   make olddefconfig
   

   Done when: the fragment applies cleanly and the build boots under QEMU.

Failure and recovery

Output

1. Test sources at the chosen layer.
2. Kconfig fragment and config commands.
3. Run transcript with pass, fail, and skip counts, plus minimized reproducers from fuzzing.