ASLR and DEP aren’t just security features; they’re fundamental shifts in how programs interact with memory, making the very act of predictable memory access a liability.

Let’s see this in action. Imagine a simple C program designed to be vulnerable:

#include <stdio.h>
#include <string.h>

void vulnerable_function() {
    char buffer[64];
    printf("Enter text: ");
    gets(buffer); // DANGEROUS: gets() does not check buffer size
    printf("You entered: %s\n", buffer);
}

int main() {
    vulnerable_function();
    return 0;
}

Without ASLR and DEP, an attacker could craft an input that overwrites the return address on the stack, pointing it to malicious shellcode they’ve injected into the program’s memory. DEP (Data Execution Prevention) would prevent the shellcode from running if it’s placed in a data segment, but ASLR (Address Space Layout Randomization) is what makes guessing the location of that shellcode, or any other critical code segment, a nightmare for the attacker.

Here’s how you enable them on a modern Linux system.

Address Space Layout Randomization (ASLR)

ASLR randomizes the memory addresses where executables, libraries, stack, and heap are loaded. This means that each time a program runs, its memory layout is different. An attacker can’t rely on a fixed address for a function or variable.

Enabling ASLR:

ASLR is typically controlled by a kernel parameter. You can check its current state:

cat /proc/sys/kernel/randomize_va_space
  • 0: ASLR is disabled.
  • 1: ASLR is enabled, but with some limitations (e.g., stack and mmap are randomized, but heap is not).
  • 2: Full ASLR is enabled (stack, mmap, heap, and libraries are randomized).

To enable full ASLR, you’ll want to set it to 2. You can do this temporarily:

sudo sysctl -w kernel.randomize_va_space=2

For a permanent change, edit /etc/sysctl.conf and add or modify the line:

kernel.randomize_va_space = 2

Then, apply the changes:

sudo sysctl -p

Why it works: When kernel.randomize_va_space is set to 2, the kernel modifies the base addresses of the executable, shared libraries, the stack, and the heap every time a process starts. This forces an attacker to first find the address of a specific piece of code or data (e.g., a "gadget" for return-oriented programming) within the current execution’s memory map, which is extremely difficult if that map changes with every run.

Data Execution Prevention (DEP)

DEP, often implemented as NX (No-Execute) or XD (eXecute Disable) bit on modern processors, marks memory pages as either executable or non-executable. If the CPU tries to execute code from a non-executable page, it triggers an exception, usually terminating the program.

Enabling DEP:

DEP is largely a hardware feature, controlled by the NX/XD bit in your CPU. Linux kernels leverage this. For DEP to be effective, two things must be true:

  1. CPU Support: Your CPU must support the NX/XD bit. Most modern CPUs do. You can check this by looking at your CPU flags:

    grep -i nx /proc/cpuinfo
    

    If you see nx in the output, your CPU supports it.

  2. Kernel Support and Configuration: The Linux kernel must be compiled with NX support and enabled at boot. This is standard on virtually all modern distributions. The page_exec setting in /proc/sys/vm/ can influence execution permissions, but the primary mechanism is the hardware NX bit. When the kernel maps memory, it sets the appropriate permissions. For example, memory allocated for data (like the stack or heap) is marked as non-executable.

How to verify DEP is active:

You generally don’t "enable" DEP via a sysctl knob like ASLR. If your CPU supports NX and your kernel is modern, it’s active by default for memory regions that shouldn’t contain executable code.

You can observe its effect with tools like execstack:

# Compile the vulnerable program without PIE (Position-Independent Executable)
gcc -fno-stack-protector -z execstack -o vulnerable_no_dep vulnerable.c

# Compile the vulnerable program with DEP enabled (standard behavior)
gcc -fno-stack-protector -o vulnerable_with_dep vulnerable.c

Now, try to run shellcode. If DEP is effective, attempting to execute code from a data segment (like the buffer in vulnerable.c if an attacker could overwrite the return address to point there) will cause a segmentation fault with an IO_SEGV error.

  • vulnerable_no_dep (compiled with -z execstack): Might allow execution from the stack if an exploit is successful.
  • vulnerable_with_dep (compiled without -z execstack): The stack segment is marked non-executable by default. If an attacker manages to overwrite the return address to point to shellcode on the stack, the CPU will refuse to execute it, and the program will crash with a segmentation fault (SIGSEGV).

Why it works: DEP ensures that memory regions designated for data storage (like the stack and heap) cannot be executed as code. This directly thwarts attacks where an attacker injects malicious code into these data areas and tries to trick the program into running it.

The Combined Power:

Together, ASLR and DEP create a formidable defense. ASLR makes it incredibly hard for an attacker to know where to jump to execute their code, and DEP prevents them from executing code if they manage to get it into the wrong memory segment.

The next hurdle you’ll encounter after enabling these is dealing with more sophisticated exploit techniques that bypass these protections, such as return-to-libc attacks or advanced ROP (Return-Oriented Programming) chains that chain together existing executable code snippets without injecting new code.

Want structured learning?

Take the full Cdk course →