Os Dev Scratch

OS development from scratch

Contract

Inputs

1. Target ISA and firmware (required): x86-64 with BIOS/legacy or UEFI; xv6-RISC-V as a reference port.
2. Host toolchain (required): a cross toolchain and QEMU available or installable.
3. Milestone (optional): boot, long mode, interrupts, or scheduling.

Procedure

1. Pick the boot path and stick to it.

   BIOS path (legacy)
   +-- POST
   +-- MBR boot sector loads stage 2
   +-- GRUB or another multiboot loader loads the kernel ELF
   +-- kernel entry (_start)
   
   UEFI path (modern)
   +-- UEFI firmware
   +-- EFI loader (limine, systemd-boot) loads kernel from the ESP
   +-- handoff with the memory map
   

   Multiboot keeps the first milestone short; UEFI is the path for real hardware. Done when: one path is named and the loader's load address is known.

2. Stand up the freestanding cross toolchain. Use x86_64-elf-gcc and binutils, not the host compiler: the host toolchain injects startup files and libc assumptions that break a kernel.

   x86_64-elf-gcc -ffreestanding -nostdlib -c kernel.c -o kernel.o
   x86_64-elf-ld -T linker.ld kernel.o -o kernel.elf
   

   ENTRY(_start)
   SECTIONS {
       . = 0x100000;   /* 1 MiB, the multiboot convention */
       .text   : { *(.text .text.*) }
       .rodata : { *(.rodata .rodata.*) }
       .data   : { *(.data .data.*) }
       .bss    : { *(.bss .bss.*) }
   }
   

   The linker script's load address must match what the loader promises; a mismatch is the classic relocation triple fault. Done when: kernel.elf links and readelf -h shows the expected entry and load address.

3. Boot under QEMU with serial as the first output.

   qemu-system-x86_64 -kernel kernel.elf -serial stdio -m 128M -no-reboot -no-shutdown
   

   -kernel loads an ELF carrying a multiboot header; the header must sit in the first 8 KiB of the image. For the UEFI path, run under OVMF with a disk image the loader has claimed:

   qemu-system-x86_64 -bios /usr/share/ovmf/OVMF.fd -drive file=disk.img,format=raw -serial stdio
   

   Done when: the first serial_putc('K') from the entry path shows in the terminal.

4. Enter long mode through PAE and 4-level paging. The path is: protected mode, enable PAE, build PML4/PDPT/PD, set EFER.LME, then enable paging.

   uint64_t pml4[512] __attribute__((aligned(4096)));
   uint64_t pdpt[512] __attribute__((aligned(4096)));
   uint64_t pd[512]   __attribute__((aligned(4096)));
   
   void setup_paging(void) {
       for (int i = 0; i < 512; i++)
           pd[i] = (uint64_t)i * 0x200000 | 0x83;  /* 2 MiB pages, present|rw|PS */
       pdpt[0] = (uint64_t)pd   | 0x03;
       pml4[0] = (uint64_t)pdpt | 0x03;
       __asm__ volatile("mov %0, %%cr3" :: "r"(pml4));
   }
   

   This identity-maps the first GiB with 2 MiB pages; the kernel stack must be valid before any interrupt can fire. Done when: code runs at 64-bit with serial output intact.

5. Load the IDT and dispatch by vector. Each gate points at an assembly stub that pushes a trap frame; the C handler switches on the vector.

   struct idt_entry {
       uint16_t offset_low;
       uint16_t selector;
       uint8_t  ist;
       uint8_t  type_attr;
       uint16_t offset_mid;
       uint32_t offset_high;
       uint32_t zero;
   } __attribute__((packed));
   

   void interrupt_handler(struct trap_frame *f) {
       if (f->vector == 14)        /* page fault: cr2 holds the address */
           handle_page_fault(f->cr2, f->error_code);
       else if (f->vector == 33)   /* IRQ1 keyboard after PIC remap */
           keyboard_handler();
   }
   

   Done when: a deliberate page fault is caught, reported over serial, and execution continues or halts cleanly.

6. Remap the PIC before unmasking anything. The 8259 pair delivers IRQs at vectors 32 through 47 after remap; the APIC (LAPIC plus I/O APIC, discovered from the ACPI MADT) is the modern controller and its LAPIC timer is the preemption source.

   outb(0x20, 0x11); outb(0xA0, 0x11);  /* ICW1: init, cascade */
   outb(0x21, 0x20); outb(0xA1, 0x28);  /* ICW2: vector offsets 0x20 and 0x28 */
   

   Send EOI (0x20 to port 0x20) at the end of every handled IRQ. Done when: the timer or keyboard IRQ fires exactly once per event.

7. Write serial first, keyboard second. Poll COM1's line status for transmit-empty; the keyboard is a scancode port with an EOI.

   void serial_putc(char c) {
       while ((inb(0x3F8 + 5) & 0x20) == 0)
           ;
       outb(0x3F8, c);
   }
   
   void keyboard_handler(void) {
       uint8_t sc = inb(0x60);
       char c = scancode_to_ascii[sc];
       if (c)
           serial_putc(c);
       outb(0x20, 0x20);   /* EOI */
   }
   

   Done when: typed keys echo over serial.

8. Build the frame allocator from the firmware memory map. Take usable regions from the multiboot or UEFI memory map, mark one bitmap bit per 4 KiB frame, and return frame addresses on first fit.

   #define PAGE_SIZE 4096
   uint64_t alloc_frame(void) {
       for (uint64_t i = 0; i < total_frames; i++)
           if (!test_bit(frame_bitmap, i)) {
               set_bit(frame_bitmap, i);
               return i * PAGE_SIZE;
           }
       return 0;   /* out of frames: the caller must see this */
   }
   

   A linear scan is fine at this scale; replace it with a free list only when allocation shows up in a profile. Done when: every handed-out frame is later reclaimed exactly once.

9. Switch contexts cooperatively before adding preemption. Save callee-saved registers per the System V AMD64 ABI, restore the next task's, and return into its saved rip.

   struct context {
       uint64_t rbx, rbp, r12, r13, r14, r15, rsp, rip;
   };
   void switch_context(struct context *old, struct context *new);
   

   Caller-saved registers live in the stack frame already, so only the callee-saved set plus stack pointer and return address need storing. Add a timer-IRQ preemption tick only after cooperative switching is stable. Done when: two tasks alternate under a yield call without corrupting either stack.

10. Keep xv6-RISC-V open as the reference.

    git clone https://github.com/mit-pdos/xv6-riscv && cd xv6-riscv && make qemu
    

    xv6 component	Concept
    kernel/vm.c	Page table management
    kernel/trap.c	Trap dispatch
    kernel/proc.c	Context switch and scheduler
    kernel/plic.c	Interrupt controller
    user/usys.pl	System call stubs

    The concepts port directly to x86-64; the register and controller details do not. Done when: each new milestone reads the corresponding xv6 file first.

Failure and recovery

Output

1. The chosen boot path and load address.
2. Per-stage code patterns with the acceptance command for each.
3. The next milestone named.