In this blog post, we will walk you through the process of developing a monolithic kernel call tinyOS from scratch by following tutorials from JamesM's and Bran's Kernel Development guides. This includes setting up a cross-compilation toolchain, writing assembly and C code for the kernel, creating an ISO image with GRUB, and testing it in a virtual machine (VM) using VirtualBox. Since some resources are no longer available, we will also provide alternative solutions and details on the entire process.
There are several kernel development tutorials available on the internet, but JamesM's tutorial stands out as more organized and comprehensive, covering a broad range of features. I followed this tutorial a few years ago, but unfortunately, the original site is no longer accessible. Thankful to archive.org, the content is still available, and the full source code can also be downloaded from the site.
 |
| tinyOS in GRUB menu. |
Before diving into the kernel development, we need to ensure that the following tools are installed and properly configured on the development environment:
- Cross-Compiler toolchain (GCC and Binutils)
- GRUB (for bootloader configuration)
- NASM (for assembly code compilation)
- VirtualBox (for testing the kernel)
The key to building an operating system from scratch is cross-compilation. This process ensures the kernel is compiled in an environment separate from the target system. To create a custom cross-compilation toolchain using GCC and Binutils, follow the setup instructions below.
Before proceeding with the custom cross-compilation toolchain, install the following dependencies in the build environment.
sudo apt install build-essential bison flex libgmp3-dev libmpc-dev libmpfr-dev texinfo libisl-dev
First, we must build Binutils (the assembler and linker) for our cross-compilation toolchain. To install it, we use the following steps:
1. Download and extract the latest version of Binutils source code from the GNU website.
2. Configure Binutils to build for the i686-elf target architecture:
./configure --target=$TARGET --prefix="$PREFIX" --with-sysroot --disable-nls --disable-werror
3. Build and install Binutils to the ~/gcc-i686/cross directory.
The --target=i686-elf specifies that we want to target the i686 architecture (32-bit x86). The --prefix option sets the installation directory.
Now, we need to build GCC for cross-compilation. GCC is the compiler used to compile C code into machine code. We’ll build GCC with the i686-elf target and without generating any C libraries using the --without-headers option. This is crucial since we are building our kernel and won’t be relying on standard system libraries.
1. Download the source code for GCC.
2. Extract and configure GCC to target the i686-elf architecture:
cd build-gcc/
../gcc/configure --target=$TARGET --prefix="$PREFIX" --disable-nls --enable-languages=c,c++ --without-headers
3. Build gcc and libgcc (A support library that the compiler needs during the compile time, functioning at a low level.).
This process may take some time to complete because GCC must be compiled from the source.
After setting up the cross-compiler, we can begin kernel development. We closely followed JamesM’s and Bran’s Kernel Development tutorials, as they provide a solid foundation for writing a basic kernel.
The source codes relevant for each of the below sections are listed in the project Git repository at https://git.hub.com/dilshan/mini-monolithic-kernel.
Genesis: The Genesis section in JamesM's tutorials represents the initial step in kernel development. This phase focuses on establishing a minimal boot environment necessary for booting the operating system. The fundamental structure includes creating a simple assembly file that interfaces with GRUB, prepares the Memory Management Unit (MMU), and jumps to a basic kernel entry point. The kernel is expected to produce simple output, such as a "Hello, World!" message, displayed on the screen using direct port I/O. This provides visual confirmation that the kernel is functioning correctly. Completing this stage sets the foundation for subsequent stages, which involve memory management, interrupts, and system calls.
The Screen: The Screen section introduces the concept of outputting text to the screen, which is critical for debugging and interacting with the kernel during its early development. In this section, the tutorials show how to write a basic video driver, leveraging the VGA text mode (mode 0x0F) for text display. Using port I/O, we write characters directly to the video memory area (located at address 0xB8000 for VGA). This is accomplished by defining a function to output characters and manage the screen's cursor position. The kernel is equipped with basic screen manipulation functions to clear the screen, print strings, and position the cursor. This basic video output mechanism is foundational for debugging and visualizing the kernel's execution during development.
|
| Hello World example on VM. |
The GDT and IDT: In the GDT and IDT section, the kernel is equipped with the Global Descriptor Table (GDT) and the Interrupt Descriptor Table (IDT), which are crucial for the system's architecture. The GDT defines the memory segments that the CPU will use to access code, data, and stack segments in protected mode. The tutorials cover how to define and load the GDT, setting up descriptors for kernel and user space, along with data, code, and stack segments. The IDT is similarly vital for handling interrupts and exceptions. This section involves setting up interrupt gates and defining entries in the IDT for each exception (e.g., division by zero, page faults) and hardware interrupt (e.g., keyboard or timer interrupts). Both the GDT and IDT are initialized during the kernel's boot process, ensuring that the system has a structured and secure way to handle memory access and interrupt servicing.
int main(void *mboot_ptr) {
asm volatile ("int $0x3");
for (;;);
return 0xDEADBABA;
}
 |
| Detecting unhandled software interrupts. |
IRQs and the PIT: The IRQs and the PIT section introduces Interrupt Request Lines (IRQs) and the Programmable Interval Timer (PIT), which are used to handle real-time events and interrupt-based tasks. The tutorials guide you through setting up the PIC (Programmable Interrupt Controller) to enable IRQs, allowing the kernel to handle hardware interrupts. This includes masking specific interrupt lines to ignore certain hardware events. The PIT, in particular, is configured to generate timer interrupts at a fixed rate (e.g., every 1 ms), which is essential for maintaining time, handling process scheduling, and creating time-based events. The setup of IRQs and PIT is vital for enabling multitasking and real-time functionality in the kernel, as it allows the operating system to react to hardware events and schedule tasks efficiently.
 |
| Capturing and handling timer interrupts. |
Paging: The Paging section is critical for implementing virtual memory and managing memory in a more flexible, protected manner. In this phase, the kernel is extended with paging functionality, which allows it to use page tables to map virtual addresses to physical memory addresses. This introduces a level of abstraction between the CPU and the physical memory, allowing for things like virtual address spaces, protection of kernel memory, and the ability to swap processes in and out of memory. The tutorials cover setting up the Page Directory and Page Tables, enabling paging, and configuring the CR3 register to manage the paging mechanism. With paging enabled, the system can now handle larger applications, prevent programs from directly accessing sensitive areas of memory, and isolate process memory spaces for security and stability.
The Heap: In the Heap section, the kernel is introduced to dynamic memory allocation, enabling it to allocate and de-allocate memory during runtime. Unlike stack memory, which is allocated in fixed sizes, the heap is a region of memory where programs can request memory dynamically (using functions like kmalloc and kfree, see example code below.). This section explains how to implement a memory manager that keeps track of free and used memory blocks. The kernel can now handle the allocation of memory for objects that don't have a fixed lifetime or size, which is essential for running more complex applications and managing system resources dynamically. A robust memory manager is vital for the efficiency and scalability of the kernel as it grows.
#include "system.h"
#include "descriptor_tables.h"
#include "timer.h"
#include "paging.h"
#include "kmalloc.h"
int main(void *mboot_ptr) {
init_descriptor_tables();
init_paging();
asm volatile ("sti");
uint32_t *a = kmalloc(8);
uint32_t *b = kmalloc(8);
uint32_t *c = kmalloc(8);
*a = 10;
*b = 20;
*c = *a + *b;
printk("value = %d", *c);
kfree(c);
kfree(b);
kfree(a);
for (;;);
return 0xDEADBABA;
}
Chapter 8 of the tutorial, which covered initrd (a RAM-based file system) is no longer viable due to the unavailability of necessary modules.
Multitasking: The Multitasking section is one of the most pivotal in the kernel's development, enabling the kernel to handle multiple processes concurrently. Initially, the kernel operates in single-tasking mode, but in this section, the kernel is extended to perform multitasking by setting up simple process scheduling and context switching. The tutorial covers how to save the state of a process (e.g., CPU registers, stack pointer) when switching between tasks and how to load the state of another process when resuming execution. With multitasking, the kernel can run multiple tasks simultaneously, managing CPU resources between them, making the system much more versatile and responsive.
|
| Handling thread with main service loop. |
Once we have written the kernel code, the next step is to package it into a bootable ISO file. We will use GRUB as the bootloader, which is capable of loading the kernel from the ISO during the boot process. GRUB is compatible with BIOS-based systems, making it suitable for testing in a virtual machine (VM).
The update_image.sh script is responsible for automating this process. In the script file grub-mkrescue command is used to generate the bootable ISO with GRUB. This command takes the GRUB configuration and our kernel image and creates a bootable ISO image.
In the above script, tinyos.bin is the compiled kernel image, and grub.cfg is the GRUB configuration file that specifies how to boot the kernel.
Once the ISO image is generated, we need to test our kernel. For this, we use VirtualBox, which allows us to run virtual machines with custom ISO images.
Here are the steps to set up the VM:
1. Create a New Virtual Machine in VirtualBox:
- Select Linux as the OS type.
- Choose the Arch Linux (32-bit) sub-type.
- Allocate 8MB of base memory.
- CPU execution cap: approximately 2%.
 |
| Settings for the Virtual machine. |
2. Mount the ISO Image: In the VM settings, go to Storage and mount the tinyos.iso file as a CD/DVD drive.
3. Start the VM:
- Boot the VM, and the GRUB bootloader should load your kernel from the ISO.
- If everything is set up correctly, you should see your kernel’s output, such as a message printed on the screen.
The source code and some binaries relevant for this project can be found at https://github.com/dilshan/mini-monolithic-kernel.
This kernel serves as an entry point for building an operating system. Depending on the objectives, it can be further developed by expanding peripheral support, introducing proper process management and scheduling mechanisms, creating a file system, and developing user space libraries. The most critical features that this kernel needs to implement are the user mode and kernel mode layers, adequate memory protection, and a simple file system.
Comments