Regular Expressions

A regular expression (regex) is a pattern for searching, extracting, or validating text. In security they’re everywhere: filter tool output, extract URLs/emails/hashes from a dump, write detection rules (WAF, IDS, grep in logs), and understand —or break— input validations. Mastering regex turns megabytes of noise into the exact line you want.

Essential syntax

.        any character             \d  digit       \w  alphanumeric+_
*        0 or more                 \s  whitespace  \b  word boundary
+        1 or more                 ^   start of line
?        0 or 1 (optional)         $   end of line
{2,5}    between 2 and 5 reps       []  class: [a-z], [0-9], [^abc] (negated)
(...)    capture group             |   alternation (a|b)
\        escapes a metacharacter    (?:...) non-capturing group

Useful pentest patterns

# emails
[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}
# IPv4
\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b
# URLs
https?://[^\s"'<>]+
# MD5 hash (32 hex)
\b[a-fA-F0-9]{32}\b
# possible secrets/API keys
(?i)(api[_-]?key|secret|token|password)["'\s:=]+[A-Za-z0-9._-]{8,}

Using them in practice

# extended grep: print only the match (-o)
grep -oE 'https?://[^ "]+' file.txt | sort -u
grep -rE '(?i)password\s*=' .            # find credentials in code
# in Python
import re
re.findall(r"\b[A-Fa-f0-9]{32}\b", text)    # all MD5s
re.search(r"id=(\d+)", url).group(1)          # capture a group
# sed with capture groups (\1)
sed -E 's/(user)=[^&]+/\1=REDACTED/' log

Greedy vs lazy (a classic mistake)

<.*>     # greedy: captures too much (whole <a> ... </b>)
<.*?>    # lazy: captures the minimum (each tag separately)

The “lazy” quantifier (?) stops .* from swallowing too much — a typical source of broken regex.

Why it matters in security

Extract IOCs from a log, pull all subdomains from a recon dump, find hardcoded keys in a repo, write a detection signature, or analyze why a WAF blocks your payload: it’s all regex. And on the other side, many input validations (and WAF filters) are badly written regex you can evade — understanding them is breaking them.

Common mistakes

Forgetting to escape literal metacharacters (., (, [).
Greedy where you wanted lazy.
Catastrophically slow regex (ReDoS) with nested quantifiers (a+)+.
Validating emails/URLs with impossible “perfect” regex — sometimes “contains @ and a dot” is enough.

Mastery checklist

I know metacharacters, classes, quantifiers, and anchors
I write patterns for email, IP, URL, and hashes
I use grep -oE and re.findall/re.search to extract data
I capture and reuse groups (\1, .group(1))
I understand greedy vs lazy and when to use each
I recognize ReDoS risks and evadable validations
I test my regex in regex101 before trusting it