Level & Layer Abstraction

“Abstraction without hardware enforcement is just an illusion. The entire stability, security, and multitasking capability of Linux rests on physical CPU privilege boundaries and memory management hardware.”

At the heart of Linux engineering lies a fundamental requirement: how does the operating system allow thousands of arbitrary, untrusted, or buggy user programs to execute on physical silicon without corrupting the machine, crashing other programs, or taking over the hardware?

The answer is hierarchical abstraction, enforced by physical hardware mechanisms: CPU protection rings, the Memory Management Unit (MMU), and the system call boundary.

1. The Three Primary Architectural Levels

A running Linux system is stratified into three distinct tiers:

+-------------------------------------------------------------------------+
|                              USER SPACE                                 |
|   Web Browsers, Compilers, Daemons, Shells, Container Runtimes          |
|   Executes in: CPU Ring 3 (User Mode) / Virtual Memory                  |
+-------------------------------------------------------------------------+
                                     |
                       System Call Interface (syscall)
                                     v
+-------------------------------------------------------------------------+
|                             KERNEL SPACE                                |
|   Process Scheduler, VFS, Memory Manager, Networking, Device Drivers    |
|   Executes in: CPU Ring 0 (Supervisor Mode)                             |
+-------------------------------------------------------------------------+
                                     |
                          Hardware Bus & Registers
                                     v
+-------------------------------------------------------------------------+
|                          PHYSICAL HARDWARE                              |
|   CPU (Cores, Caches, MMU), Physical RAM, NVMe/Disks, NICs, Timers       |
+-------------------------------------------------------------------------+

1. Physical Hardware: The silicon layer. Includes execution cores, cache hierarchies (L1, L2, L3), the Memory Management Unit (MMU), physical DRAM, Interrupt Controllers (APIC), NVMe/storage controllers, and Network Interface Cards (NICs).
2. Kernel Space: The core operating system software executing with unrestricted processor privileges. It manages physical resources, handles hardware interrupts, enforces access control, and multiplexes CPU time and memory across tasks.
3. User Space: All running user applications, utilities, and background system services. Code in user space operates under strict hardware constraints with zero direct access to physical devices or raw memory.

2. Hardware Privilege Rings: Ring 0 vs. Ring 3

To prevent user processes from tampering with critical hardware, modern microprocessors enforce hardware-level privilege levels.

x86-64 Protection Rings:

The x86-64 architecture defines four privilege rings (Rings 0 through 3), shaped like concentric circles:

           +---------------------------------------+
           |  Ring 3: User Mode (Applications)     |
           |   +-------------------------------+   |
           |   |  Ring 2: (Unused in Linux)    |   |
           |   |   +-----------------------+   |   |
           |   |   | Ring 1: (Unused)      |   |   |
           |   |   |   +---------------+   |   |   |
           |   |   |   | Ring 0: Kernel|   |   |   |
           |   |   |   | (Supervisor)  |   |   |   |
           |   |   |   +---------------+   |   |   |
           |   |   +-----------------------+   |   |
           |   +-------------------------------+   |
           +---------------------------------------+

• Ring 0 (Supervisor Mode / Kernel Mode): Maximum privilege. The CPU can execute all instructions, including privileged CPU control operations (cli to disable interrupts, lidt to set interrupt descriptor tables, mov cr3, reg to switch page tables, and wrmsr to modify model-specific registers). The Linux kernel executes here.
• Ring 3 (User Mode): Minimum privilege. User programs execute here. Any attempt by a Ring 3 instruction to manipulate I/O ports directly, disable interrupts, or modify CPU control registers triggers a hardware General Protection Fault (#GP), which the kernel catches to terminate the errant process with SIGSEGV or SIGILL.
• Why are Rings 1 and 2 unused? Most RISC processor architectures (like ARM, MIPS, RISC-V) only feature two privilege levels for operating systems. To keep the Linux kernel portable across all silicon platforms, Linus designed the kernel around a clean two-level model: Kernel Mode and User Mode.

ARM64 Exception Levels:

On 64-bit ARM (AArch64), the equivalent model uses Exception Levels:

• EL0: User Space applications.
• EL1: Operating System Kernel (Linux).
• EL2: Hypervisor / Virtualization (KVM).
• EL3: Secure Monitor / TrustZone.

3. Virtual Address Space: The 128 Terabyte Split

Every process in Linux is given the illusion that it possesses its own contiguous, private address space. This is achieved via Virtual Memory orchestrated by the kernel and translated in real-time by the CPU’s Memory Management Unit (MMU).

On standard 64-bit x86-64 Linux (using 48-bit canonical addressing), the virtual address space spans 256 Terabytes, cleanly split into two halves:

0xFFFFFFFFFFFFFFFF +---------------------------------------------+
                   |                                             |
                   |           KERNEL SPACE (128 TB)             |
                   |  (Direct Mapping of RAM, vmalloc, Modules)  |
                   |                                             |
0xFFFF800000000000 +---------------------------------------------+
                   |                                             |
                   |      NON-CANONICAL "HOLE" (Empty Space)     |
                   |    (Access triggers hardware #GP fault)     |
                   |                                             |
0x00007FFFFFFFFFFF +---------------------------------------------+
                   |                                             |
                   |            USER SPACE (128 TB)              |
                   |    Stack, Memory Mappings (mmap), Heap,     |
                   |            BSS, Data, Text (Code)           |
                   |                                             |
0x0000000000000000 +---------------------------------------------+

Key Memory Guarantees:

1. Isolation Between Processes: Process A cannot read or write to Process B’s memory. Even if both processes use the virtual pointer address 0x00400000, the MMU page tables map that virtual address to completely different physical RAM frames.
2. Kernel Protection: User-space code executing in Ring 3 cannot access addresses in the upper half (0xFFFF800000000000+). The MMU checks the User/Supervisor (U/S) bit in the page table entries; an illegal access triggers a Page Fault (#PF), which the kernel translates into a segmentation fault.
3. KPTI (Kernel Page Table Isolation): Introduced to mitigate the hardware speculative execution vulnerability Meltdown, KPTI maintains two separate page tables for each process: one with kernel mappings for Ring 0, and a restricted page table with minimal kernel trampolines for Ring 3.

4. The Transition Mechanism: System Calls (syscalls)

User-space applications cannot touch disk drives, transmit network packets, or allocate physical RAM directly. To interact with the physical world, user programs must request service from the kernel via a System Call (syscall).

 [ User Process (Ring 3) ]
             |
             |  1. Invokes glibc wrapper: write(1, "hello", 5)
             v
 [ GNU C Library (glibc) ]
             |
             |  2. Loads syscall number into %rax (e.g., __NR_write = 1)
             |  3. Loads arguments into %rdi, %rsi, %rdx
             v
 [ CPU Instruction: syscall ]  <--- HARDWARE PRIVILEGE TRANSITION (Ring 3 -> Ring 0)
             |
             v
 [ Linux Kernel Entry: entry_SYSCALL_64 ]
             |
             |  4. Saves user registers to struct pt_regs
             |  5. Switches from User Stack to Kernel Stack
             |  6. Looks up sys_call_table[__NR_write]
             v
 [ VFS / Device Subsystem ]
             |
             |  7. Executes vfs_write(), transfers data to tty/driver
             v
 [ CPU Instruction: sysret ]   <--- HARDWARE RETURN (Ring 0 -> Ring 3)
             |
             v
 [ User Process Resumes ]

Hardware Register Convention (x86-64):

When transitioning via the syscall instruction:

• %rax: Holds the System Call Number (e.g., 0 for read, 1 for write, 2 for open, 56 for clone). Upon return, %rax holds the return value or negative error code (e.g., -EPERM).
• %rdi: First argument.
• %rsi: Second argument.
• %rdx: Third argument.
• %r10: Fourth argument.
• %r8: Fifth argument.
• %r9: Sixth argument.

The Cost of a System Call:

A function call inside user space costs ~1-3 CPU cycles (a simple branch instruction). A system call costs ~50-100+ CPU cycles because the CPU must validate privilege states, switch stack pointers, save architectural registers, and clear speculative branch predictors.

5. Fast Path: vDSO and vsyscall

For certain extremely frequent system calls that merely read kernel state—such as gettimeofday(), time(), and clock_gettime()—performing a full hardware privilege switch into Ring 0 imposes unacceptable latency overhead.

Linux solves this via the vDSO (Virtual Dynamic Shared Object):

1. The kernel creates a small, read-only memory page containing kernel timing data and exported routines.
2. During process startup (execve), the kernel maps this page directly into the process’s user-space address space.
3. When glibc calls clock_gettime(), it detects the vDSO mapping and executes the calculation entirely in Ring 3 by reading the CPU’s invariant Time Stamp Counter (rdtsc), avoiding the context switch to Ring 0.

You can view the active vDSO mapping for any process:

cat /proc/self/maps | grep -E 'vdso|vsyscall'

6. The C Library & Dynamic Linker (ld-linux.so)

In How Linux Works, Brian Ward clarifies that user applications almost never invoke the syscall instruction directly. Instead, they interact with the C Standard Library (glibc or musl), which serves as the essential bridge between user space and the kernel.

+-------------------------------------------------------------------------+
|                  USER APPLICATION: printf(), malloc()                   |
+-------------------------------------------------------------------------+
                                    |
+-------------------------------------------------------------------------+
|    GNU C LIBRARY (glibc): Formats text, allocates arena heap,           |
|    loads syscall number, sets registers, checks errno                   |
+-------------------------------------------------------------------------+
                                    |
                      [ CPU syscall instruction ]
                                    v
+-------------------------------------------------------------------------+
|    LINUX KERNEL: sys_call_table[__NR_write], sys_brk, sys_mmap          |
+-------------------------------------------------------------------------+

The C Library’s Three Responsibilities:

1. System Call Wrappers: Translates standard C functions into raw kernel system calls. It copies the kernel return value into the global errno variable and returns -1 on error.
2. User-Space Buffering: Standard I/O streams (FILE*, fprintf(), fread()) maintain internal 4KB or 8KB buffers in user space, aggregating small byte operations before issuing expensive write() or read() system calls to the kernel.
3. Memory Management: Functions like malloc() and free() manage user-space heaps, requesting memory in large chunks from the kernel via brk() or mmap() to avoid trapping into Ring 0 on every individual allocation.

Dynamic Linking & Symbol Resolution:

When you execute a dynamically linked ELF binary, the kernel does not jump directly to main(). Instead:

1. The kernel reads the ELF PT_INTERP segment, which points to the Dynamic Linker: /lib64/ld-linux-x86-64.so.2.
2. The kernel loads both the application and the dynamic linker into user memory, transferring execution to the linker.
3. The dynamic linker resolves all shared libraries (.so) needed by the binary:
   • It checks DT_RPATH / DT_RUNPATH embedded inside the ELF.
   • It checks the LD_LIBRARY_PATH environment variable.
   • It consults the binary cache /etc/ld.so.cache (regenerated by running sudo ldconfig).
   • It defaults to trusted system paths: /lib64, /usr/lib64.
4. It sets up the Procedure Linkage Table (PLT) and Global Offset Table (GOT) for runtime lazy symbol resolution before jumping to _start and main().

7. Kernel Threads: What Runs Inside Ring 0?

In addition to serving user-space system calls, the Linux kernel must perform background maintenance: writing dirty filesystem buffers to disk, rebalancing process queues across CPU cores, and handling software interrupts.

These background routines are executed by Kernel Threads:

• Kernel threads run exclusively in Ring 0.
• They have no user-space address space (current->mm == NULL).
• They are visible in the process table with names enclosed in square brackets (e.g., [kthreadd], [kworker/0:0], [ksoftirqd/0], [kswapd0]).
• kthreadd (PID 2) is the parent of all kernel threads, spawned during kernel initialization in init/main.c.

# List all active kernel threads on your system
ps -ef | grep '\['

8. Hands-On Practical Labs

Lab 3.1: Raw Inline Assembly System Call in C (Zero Libc Overhead)

In this lab, we write a standalone C program that bypasses the GNU C Library (glibc) entirely. Instead of calling printf() or write(), it loads registers directly and executes the hardware syscall instruction.

Source Code: raw_syscall.c

/*
 * Compile with: gcc -nostdlib -static -O2 raw_syscall.c -o raw_syscall
 * -nostdlib instructs the compiler not to link glibc or standard startup files.
 */

#define SYS_write 1
#define SYS_exit  60

/* Direct raw syscall for write(fd, buf, count) */
static inline long sys_write(int fd, const void *buf, unsigned long count) {
    long ret;
    __asm__ volatile (
        "movq %1, %%rax\n\t"  /* Syscall number (1 = SYS_write) into RAX */
        "movq %2, %%rdi\n\t"  /* Arg 1: fd (1 = stdout) into RDI */
        "movq %3, %%rsi\n\t"  /* Arg 2: buffer pointer into RSI */
        "movq %4, %%rdx\n\t"  /* Arg 3: length into RDX */
        "syscall\n\t"         /* Execute hardware trap into Ring 0 */
        "movq %%rax, %0\n\t"  /* Return value from RAX into ret */
        : "=r" (ret)
        : "i" (SYS_write), "r" ((long)fd), "r" (buf), "r" (count)
        : "%rax", "%rdi", "%rsi", "%rdx", "%rcx", "%r11", "memory"
    );
    return ret;
}

/* Direct raw syscall for exit(status) */
static inline void sys_exit(int status) {
    __asm__ volatile (
        "movq %0, %%rax\n\t"  /* Syscall number (60 = SYS_exit) into RAX */
        "movq %1, %%rdi\n\t"  /* Arg 1: status into RDI */
        "syscall\n\t"
        :
        : "i" (SYS_exit), "r" ((long)status)
        : "%rax", "%rdi", "%rcx", "%r11", "memory"
    );
}

/* Entry point required by the ELF loader when -nostdlib is used */
void _start(void) {
    const char msg[] = "[+] Hello directly from Ring 3 via raw x86_64 syscall!\n";
    sys_write(1, msg, sizeof(msg) - 1);
    sys_exit(0);
}

Compilation & Verification:

# Compile completely standalone without standard C runtime
gcc -nostdlib -static -O2 raw_syscall.c -o raw_syscall

# Inspect binary size: Notice it is only ~8 Kilobytes!
ls -lh raw_syscall

# Verify with strace: Only two raw syscalls occur: write() and exit()
strace ./raw_syscall

Lab 3.2: Inspecting Canonical Virtual Address Space & Page Maps

Inspect the active memory layout of a process to observe user space address boundaries, the stack, heap, and vDSO mapping:

# Run a long-running process in background
sleep 300 &
PID=$!

# Inspect virtual address mappings
cat /proc/$PID/maps

Example Output Analysis:

00400000-0040c000 r-xp 00000000 08:01 123456   /usr/bin/sleep  (Code .text segment)
0060b000-0060c000 r--p 0000b000 08:01 123456   /usr/bin/sleep  (Data .data segment)
01234000-01255000 rw-p 00000000 00:00 0        [heap]          (Dynamic heap via brk)
7f89ab100000-...  r-xp ...                     /lib/libc.so    (glibc shared library)
7ffe56780000-...  rw-p 00000000 00:00 0        [stack]         (User stack growing downwards)
7ffe567fd000-...  r-xp 00000000 00:00 0        [vdso]          (Virtual Dynamic Shared Object)

Kill the background sleep:

kill $PID

Lab 3.3: Intercepting System Library Calls via LD_PRELOAD

The dynamic linker supports LD_PRELOAD, allowing superusers and security tools to intercept library calls before standard libraries are searched.

1. Write the Hook Library: hook_puts.c

#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>

/* Intercept the standard puts() library function */
int puts(const char *str) {
    /* Function pointer to the original puts() in libc */
    static int (*orig_puts)(const char *) = NULL;
    if (!orig_puts)
        orig_puts = dlsym(RTLD_NEXT, "puts");

    orig_puts("[INTERCEPTED by LD_PRELOAD]:");
    return orig_puts(str);
}

2. Compile and Test:

# Compile as a shared object library
gcc -fPIC -shared hook_puts.c -o hook_puts.so -ldl

# Test with standard utilities
LD_PRELOAD=./hook_puts.so echo "Testing dynamic linker interception"

9. Summary Checklist

• Protection Rings: Mastered the distinction between Ring 0 (Supervisor) and Ring 3 (User).
• Canonical Address Space: Verified the 128TB split and non-canonical hole.
• Page Tables & MMU: Understood multi-level page table walks and TLB caches.
• The C Library Bridge: Traced how glibc wraps raw syscalls, manages user-space I/O buffering, and sets errno.
• Dynamic Linker: Explored /lib64/ld-linux-x86-64.so.2, /etc/ld.so.cache, and LD_PRELOAD function interception.
• Raw System Call: Executed hardware syscall instruction directly from inline assembly in C.
• Memory Mapping: Analyzed /proc/[PID]/maps to locate .text, [heap], [stack], and [vdso].