Learning path — Rabbit

Rabbit exists to be read. This document is a route through the code, with the experiments worth doing at each stop and the sources worth reading next.

Before you start

Three documents:

docs/BOOT.md explains how the CPU reaches our code and what state it is in. Read it once, then again when you touch the trampoline.
docs/ARCHITECTURE.md explains the decisions and why they were taken.
docs/BENCHMARKING.md explains how claims will be measured, so you know what the code is for.

How to read the kernel

The boot path is the spine of the project. Read it in this order; every file is small on purpose.

linker.ld - where the kernel is placed in memory, which sections exist, and which symbols the Rust code imports.
rabbit-kernel/src/trampoline.s - the 32-bit entry. It receives a pointer to the boot information in a register, builds a page table and switches the processor to long mode.
rabbit-bootinfo/src/lib.rs - the pure parser for the boot handover. No unsafe in the whole crate: this is the model to follow for anything that can be modelled as data.
rabbit-kernel/src/main.rs - the 64-bit entry and the boot sequence.
rabbit-kernel/src/gdt.rs and rabbit-kernel/src/idt.rs - the descriptor tables and the interrupt path.
rabbit-kernel/src/isrs.s - the assembly stubs, generated by scripts/gen-isrs.sh. Read the generated file, then read the generator.
rabbit-kernel/src/pic.rs and rabbit-kernel/src/pit.rs - the two interrupt sources.
rabbit-mm/src/buddy.rs and rabbit-mm/src/heap.rs - the physical frame allocator and the kernel heap.
rabbit-kernel/src/memory.rs - how the pieces are wired together, and where the reserved memory comes from.

Experiments

Each of these breaks something on purpose and asks you to explain the serial log. The log is the instrument.

Change the UART divisor in rabbit-drivers/src/uart.rs and watch the output turn into garbage. Then explain why the divisor latch needs the DLAB bit.
Remove the EOI in the PIC handler and watch the timer stop after the first interrupt. Then explain why an unacknowledged interrupt blocks the next one.
Change the PIT divisor so the tick rate is 1 Hz, and measure how long the 100 ticks take.
Corrupt one byte of the frame metadata in the buddy allocator and see which self test catches it and which does not. Then add a test that does.
Free the same frame twice in the buddy allocator and check that the error is reported instead of silently corrupting the free lists.
Reserve one page less in the memory manager and see whether the boot still works. Then reserve one page more and watch the free frame count.
Divide by zero outside the registered trigger and observe that the kernel stops instead of looping forever.
Break the PVH note (change the descriptor in trampoline.s) and read the QEMU error. This is the fastest way to learn why the note exists.
Change the preemption interval in rabbit-kernel/src/sched.rs from one tick to ten and watch the interleaving in the serial log change. Then set it to zero and explain why the scheduler stops preempting.
Make a thread hold the console lock and yield inside the critical section. Predict the deadlock before you run it, then find the rule in docs/ARCHITECTURE.md that forbids it.
Take the same lock twice in one thread and see the recursion guard report it instead of hanging.
Change the thread stack size in rabbit-kernel/src/thread.rs to 4 KiB and give one thread a deep call chain. Find out how much room the interrupt frame needs on top of the thread stack.
Add a fourth thread and watch the run queue handle it. Then spawn nine and read what happens when the fixed size queue is full.
Change the user program in rabbit-userland to return a different exit status and watch the kernel report it. Then make it call a system call that is not implemented and read the -ENOSYS path.
Break the SYSRET selectors in the GDT module and see what the compile time assertions say. Then read the comment about why SYSRET derives the user selectors from STAR.
Make the user program execute an invalid instruction and check that the kernel reports the exception and stops instead of hanging.
Add a system call to the dispatcher: pick a number, define the semantics, return a negative errno on failure, and add it to the table in docs/ARCHITECTURE.md.
Remove the futex timeout and let a program wait forever. Then read the comment about why the timer hook has to expire the deadline.
Make the user program print an unimplemented system call number and watch the kernel report it in the serial log.
Change the userland to use a std::sync::Mutex across two threads and watch the futex path work for real.
Change the load base of the user program and watch the entry point move with it. The boot log prints the base and the real entry separately on purpose.
Make the page fault handler allocate a frame twice for the same address and check that the second fault finds a present entry and does not replace it. The protection violation path exists for exactly that.
Map a region and never touch it: check that the free frame count does not move. That is demand paging doing its job.
Change the mapping table size in rabbit-mm/src/mapping.rs and find the limit at which mmap starts failing.
Compare the weights workload before and after demand paging: the tail measurement used to measure the clock, and now it measures the fault. The difference between those two numbers is what the rig is for.

Experiments

One of them is worth repeating, because the kernel that fails it looks perfectly healthy until a symbol moves:

Put a structure that is larger than the stack of a ring 3 thread on the path of a system call. The stack is sixteen kilobytes. Watch what the linker put above it with nm and the map of the build: the interrupt descriptor table and the tables of the direct map are in the same section, and the overflow reaches them. The boot fails with a general protection fault in a place that has nothing to do with what changed, and the fault is in the delivery of the interrupt, not in the code that overflowed. A hardware watchpoint on the write that does it names the copy in one shot.

Debugging

Serial log first: it is deterministic, and the kernel prints enough to reconstruct what happened.
QEMU stops the CPU at reset with -s -S; attach with lldb and gdb-remote 1234 on macOS, or gdb and target remote :1234 on Linux.
-d guest_errors makes QEMU report invalid memory accesses.
isa-debug-exit gives the boot test a deterministic exit status, so CI can tell “the kernel booted” from “the kernel hung”.

Sources worth reading next

The Xen PVH boot protocol, the document behind docs/BOOT.md: https://xenbits.xen.org/docs/unstable/misc/pvh.html
The QEMU side of the handover, to see what the loader guarantees: hw/i386/x86-common.c in the QEMU source, function read_pvh_start_addr.
Intel Software Developer Manuals, volumes 2 and 3: the chapters on protected mode, paging, the GDT, the TSS, the IDT and the interrupt handling. Volume 3 chapter 11 (interrupts and exceptions) is the one to read next.
Linux kernel source, as the reference implementation of the ABI this project wants to match: arch/x86/kernel/head64.c for the page table switch, arch/x86/kernel/traps.c for the exception entry code, arch/x86/include/asm/desc.h for the descriptor layouts, include/uapi/asm/unistd_64.h for the system call numbers.
Writing an OS from scratch, by Philipp Oppermann, for a different style of the same material: https://os.phil-opp.com/
The Rust for Linux project, for how Rust is being brought into mainline: https://rust-for-linux.com/

How to read the userland

The kernel is the older half of the project and the easier one to read. The userland is where the purpose of the kernel shows: a model is served from an initramfs by a program that was not written for this kernel, and an agent asks that model what to do.

rabbit-infer/src/safetensors.rs - the reader of the weight file. It is a parser for a fixed shape, written without a dependency, and it is the model to follow for anything that can be modelled as data.
rabbit-infer/src/matvec.rs - the arithmetic: a blocked matrix by vector product over bfloat16 weights, with an AVX2 and FMA path, a NEON path and a scalar path that all agree to a tolerance.
rabbit-infer/src/model.rs - the decoder: RMSNorm, rotary embeddings, grouped query attention, the gated feed forward block and the key value cache. The dimensions come from the config, not from the code.
rabbit-infer/src/runtime.rs - the generation loop, and the Server that keeps the weights mapped and pinned between the calls of an agent.
rabbit-infer/src/grammar.rs - the byte level automaton that decides which tokens may follow, so the answer of the model is always a well formed tool call.
rabbit-infer/src/bpe.rs - the byte pair merge encoder, written from scratch: the pre-tokenizer of GPT-2 and the merges of the checkpoint, in one flat buffer, because the guest gives the program a small heap.
rabbit-agent/src/main.rs - the loop: prompt, tool call, tool, result, prompt. Five tools, and the model chooses which one to call.

What to read next in this repository

The roadmap issues of the tracker carry the remaining milestones, each one with its exit criterion and the way it is verified:

Issue 12, M10: the network, so the agent can call an API.
Issue 13, M11: quantization, so a model fits in the memory of the machine.
Issue 14: a checkpoint that closes the conversation, so the agent answers instead of asking again.
Issue 15: the tools that need a file system: write_file and list_dir.
Issue 10: a host with hardware acceleration, for the published numbers.
Issue 16: the deferred work, named so it is not lost: processes, signals, dynamic linking and a shell test suite.

M4 was the milestone where the project stopped being a kernel that boots and became a kernel that runs other people’s software. Everything before it was the preparation for that, and everything after it, the runtime and the agent, is the reason for it.