Reading width
Wide uses the full column for everything, text, diagrams, code, and exercises. Narrow keeps the standard reading width.
Text size
Scales the body text. Headings and code blocks keep their size.
In this section
0.2 Fuzzing Fundamentals
Introduction
Fuzzing finds bugs by running a program on a very large number of inputs nobody wrote by hand and watching for failures. It is one of the most productive ways to find memory corruption, parser bugs and crashes, it is how a great deal of vulnerability research is done, and Module 2 builds a fuzzer whose inputs a model helps choose.
Before a model can make a fuzzer better, you need to know what a fuzzer is made of, what it can and cannot reach, and how its results are judged.
This lesson builds a mutation fuzzer from nothing against a small parser with bugs planted in it. It starts with the target and the rule that decides what counts as a crash, has an assistant write the fuzzer to a contract, and runs it.
Then it measures three things: what happens when the crash rule is too loose, how much of the target twenty thousand executions actually reach, and what a single well-chosen seed does for a bug random changes never find. That last measurement is the question Module 2 is built on.
The fuzzer owns the inputs and the record of what broke; the target owns the oracle. A ValueError is the parser refusing input as designed, so only other exceptions reach the crash table.
The Target and Its Promise
What counts as a bugThe target is a parser for a small record format, written for this lab. Its docstring is its specification.
⋯ 5 lines folded6 0x07 checksum a two-byte big-endian sum of every value byte before it, modulo 655367 0x09 name a user name; an administrator's name is ADMIN and a priority digit from 1 to 989parse(data) returns the records as a list of (type, value). Input this format does not allow is rejected with ValueError:10that is the parser working, not a crash. Any other exception is a bug.11"""12import struct1314MAGIC = b'RC'151617def parse(data):18 if not isinstance(data, (bytes, bytearray)) or data[:2] != MAGIC:19 raise ValueError('not an RC stream: missing magic')20 out, i, total = [], 2, 021 while i < len(data):⋯ 28 lines folded
The most important line in the docstring is the one that says what a bug is. The parser promises to reject input it does not accept by raising ValueError. That is the parser working: a malformed record is supposed to be refused, and refusing it is correct.
Any other exception means the parser hit a state its author did not handle, which is a bug. A fuzzer needs a rule like this, usually called an oracle, that decides which outcomes are findings, and the rule comes from the target's specification, not from the fuzzer.
Try the rule by hand on four inputs: a valid stream, one with the wrong magic bytes, one with an unknown record type, and one with a checksum record that is one byte short.
$ python3 try.py
b'RC\x01\x05hello' parsed [(1, 'hello')]
b'XX\x01\x05hello' rejected not an RC stream: missing magic
b'RC\x05\x01!' rejected unknown record type 0x05 at offset 2
b'RC\x07\x01\x00' CRASH error: unpack requires a buffer of 2 bytes
The first parses. The second and third are rejected, with messages saying why, which is the promise being kept. Notice that the messages are specific: they say what was wrong and where. A rejection that only said invalid input would still keep the promise, and would make every rejection look the same to a person reading a fuzzer's output.
The fourth crashes. The checksum record is meant to carry two bytes, and the parser unpacks two bytes without first checking that two are there. That is a planted bug, and it is the kind fuzzing finds well: a length the code trusts and the input controls.
The crash's exception type reads simply as error, which is the name the struct module gives its own exception class.
It is easy to overlook in a list of crashes because it looks like a generic word rather than a type, and it is a good reminder that an oracle should be defined by what is allowed, the one rejection type the specification promises, rather than by a list of what is not. A list of forbidden exception types written from memory would almost certainly have missed this one.
Authorization First
What you may fuzzThe brief in section 4 says the target was written for this lab and that testing it is authorized. That sentence is not a formality.
A fuzzer sends a target inputs designed to make it fail, at a rate no person could, and pointed at a system you do not own or have written permission to test, it is an attack whatever the intent. Every target in this course is one the lab supplies, runs on your own machine, and exists to be tested.
The same discipline applies to how a fuzzer runs. A target that crashes may also hang, exhaust memory, or write files, so fuzzing runs in an environment where those failures are contained: a disposable directory, a process with limits, a machine whose state does not matter.
The lab's targets are pure functions with no side effects, which is why the exercise can run them in-process. Module 2's targets are larger, and its lessons run them the way a real campaign would. Module 12 returns to the same question from the other side: what you may do with a bug once you have found it.
Write the authorization down before the first execution, in a file that travels with the campaign. It names the target and its version, who owns it, who authorized the testing and when, where the fuzzer runs, and what happens to what it finds.
1# Fuzzing scope: SEC410 lab23Target lab/foundations/targets/tlv.py, as shipped in the lab4Owner the SEC410 course; written to be tested5Authorized every student, on their own machine6Where in-process, on your own machine, no network7Findings stay in your records; there is no vendor to notify89Out of scope: any parser, service or file format you did not10receive in the lab pack, including ones that look the same.
The lab's scope file is short because the lab's situation is simple.
A real one is the document you point to when someone asks why a test environment fell over at three in the morning, or why a crash report reached the vendor before the press. It also constrains the assistant: an assistant asked to help fuzz a target should be given the scope file, and a target outside it is a request to decline.
The Contract
Two functions, stated before they are writtenThe exercise file states what the fuzzer must do.
1"""Lesson 0.2: a mutation fuzzer. This file is the exercise: write mutate() and fuzz().23A fuzzer runs a target over and over on inputs it generates, and records the inputs that make the target fail in a way4it should not. A mutation fuzzer generates each input by changing a known-good one, a seed, slightly.56Contract, checked by `python -m lab check f02`:78 mutate(data, rng) -> bytes9 One random change to data: flip one bit, replace one byte, insert one byte or delete one byte, chosen with rng10 (a random.Random). Never returns the input unchanged unless it is empty and the change was a deletion.1112 fuzz(target, seeds, iterations, rng) -> {'executions': int, 'crashes': {signature: crash}}13 Run target(input) iterations times. Each input is mutate() applied to a seed chosen with rng. ValueError is the14 target's documented way of rejecting bad input: not a crash. Any other exception is a crash.15 signature the root cause as a string: 'ExceptionType at file.py:line', the innermost frame of the traceback16 crash {'input': the first input that hit it, as hex, 'count': how many inputs hit it,17 'first_at': the execution number (from 1) of the first}1819The same seeds, iterations and rng seed must give the same result.20"""212223def mutate(data, rng):24 raise NotImplementedError('lesson 0.2: write mutate')252627def fuzz(target, seeds, iterations, rng):28 raise NotImplementedError('lesson 0.2: write fuzz')
Two parts of the contract carry most of the weight. The first is the crash signature: a string naming the exception type and the file and line where it was raised. Twenty inputs that crash at the same line usually share one root cause, and a fuzzer that reported them as twenty crashes would bury the one fact that matters.
The second is determinism: the same seeds, iterations and random number seed must give the same result. A crash nobody can reproduce cannot be fixed, and a campaign whose result changes on every run cannot be compared with the next one.
Run the check against the file as shipped.
$ python3 -m lab check f02
f02 0.2 Fuzzing Fundamentals: lab/foundations/fuzz.py
FAIL mutate() is written
lesson 0.2: write mutate
FAIL
The check needs both functions and stops at the first one missing. Once they exist it tests each property separately, so a fuzzer that finds the bug but cannot reproduce it, or reproduces it but changes from run to run, is told exactly which property failed.
That matters more for fuzzers than for most code, because a fuzzer that looks like it works and does not is worse than none: it reports a target clean.
Before the brief, here is the procedure the contract describes, drawn as the steps and decisions the code has to make.
Each execution changes a seed by one byte and runs the target. A ValueError is the target refusing bad input as designed; any other exception is a crash, grouped by where it was raised.
Keep it open while reading the draft: each decision in the diagram should appear as a condition in the code, and each side branch as a case the check can test.
Session: The Brief
Write it blind to the targetThe brief points at the contract and adds two constraints of its own.
Write mutate() and fuzz() in fuzz.py to the contract in its docstring. The target is targets/tlv.py, a parser written for this lab with bugs planted in it, so we are authorized to test it; don't read its code for hints beyond its docstring, because the fuzzer should work on any target with the same contract. Standard library only.
The constraint that matters is the one telling the assistant not to read the target's code for hints. An assistant given the parser's source would notice the checksum record and the ADMIN branch, and could write a fuzzer that targets them directly.
That fuzzer would pass the check and be useless on the next target. A fuzzer is a general tool, and the brief keeps it one. The standard library constraint keeps the result small enough to read in full, for the same reason the agent loop in lesson 0.1 avoided a framework.
The blind constraint is also how fuzzing is done for real. Black-box fuzzing treats the target as something you can only run, not read; grey-box fuzzing adds feedback from the running target, such as which code each input reached; white-box approaches reason over the source. This lesson's fuzzer is black-box. Section 7 measures what that costs, and the practice's stretch goal turns it grey.
The Draft
Mutate, run, sort, groupThe reply was the file. Here is the code below the docstring.
21import os222324def mutate(data, rng):25 data = bytearray(data)26 ops = ['flip', 'replace', 'insert', 'delete'] if data else ['insert']27 op = rng.choice(ops)28 if op == 'insert':29 data.insert(rng.randrange(len(data) + 1), rng.randrange(256))30 return bytes(data)31 i = rng.randrange(len(data))32 if op == 'flip':33 data[i] ^= 1 << rng.randrange(8)34 elif op == 'replace':35 data[i] = rng.choice([b for b in range(256) if b != data[i]])36 else:37 del data[i]38 return bytes(data)394041def _signature(exc):42 tb = exc.__traceback__43 while tb.tb_next:44 tb = tb.tb_next45 return f"{type(exc).__name__} at {os.path.basename(tb.tb_frame.f_code.co_filename)}:{tb.tb_lineno}"464748def fuzz(target, seeds, iterations, rng):49 crashes = {}50 for n in range(1, iterations + 1):51 data = mutate(rng.choice(seeds), rng)52 try:53 target(data)54 except ValueError:55 continue56 except Exception as e: # noqa: BLE001 - every other exception is a crash57 sig = _signature(e)58 if sig in crashes:59 crashes[sig]['count'] += 160 else:61 crashes[sig] = {'input': data.hex(), 'count': 1, 'first_at': n}62 return {'executions': iterations, 'crashes': crashes}
The mutation function picks one of four operations at random and applies it once: flip a bit, replace a byte with a different value, insert a byte, delete one. An empty input can only grow, so its only operation is insert. The replace operation chooses from every value except the current one, which guarantees the input changes, as the contract requires.
The fuzzing loop picks a seed, mutates it, runs the target, and sorts the outcome. A ValueError is skipped. Anything else is a crash, and its signature is built from the innermost frame of the traceback, the line where the exception was actually raised.
The first input for each signature is kept, as hex so it can be stored and replayed; later inputs with the same signature only add to its count.
Two choices in the draft are worth checking against the contract rather than accepting. The signature uses the innermost frame, which for this parser is the line in its own code where the exception was raised; a target that called into a library would put the library's line there, and the signature would group crashes by where the library failed rather than where the target misused it.
And every mutation is applied to a seed, never to an earlier mutant, so each input is at most one change away from a known-good one. That keeps inputs close to valid, which suits a parser with strict structure, and it limits how far from the seeds the fuzzer can ever get.
Run the check.
$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 -m lab check f02
f02 0.2 Fuzzing Fundamentals: lab/foundations/fuzz.py
ok mutate() returns bytes, one change at a time
600 of 600 changed; length within one byte: True
ok fuzz() counts its executions
3000 executions
ok the checksum bug is found
error at tlv.py:29
ok rejections (ValueError) are not crashes
1 signatures
ok every saved input reproduces its crash
all reproduce
ok the same rng seed gives the same result
ok each crash records when it was first hit and how often
error at tlv.py:29 after 2, hit 94 times
PASS
Every property holds. The check ran the fuzzer three thousand times against the parser, twice with the same random seed, and compared the results.
The checksum bug was found within the first handful of executions and hit dozens of times in three thousand, which is typical of a shallow bug: one byte in the right place triggers it, and random changes put bytes in the right place often. The saved input reproduces the crash when run again, and the same random seed gives the same result.
A Loose Oracle, Measured
Every exception a crashThe tempting simplification is to drop the rejection rule and treat every exception as a finding. More findings sounds like a better fuzzer. Here is that change.
53 target(data)- except ValueError:- continue54 except Exception as e: # noqa: BLE001 - every other exception is a crash
Run the check against it and read the crash list.
$ cp records/fuzz-noisy.py lab/foundations/fuzz.py
$ python3 -m lab check f02 | sed -n "6,10p"
ok the checksum bug is found
ValueError at tlv.py:19; ValueError at tlv.py:23; ValueError at
tlv.py:31; ValueError at tlv.py:36; error at tlv.py:29
FAIL rejections (ValueError) are not crashes
5 signatures
The one real bug is still there, now among 5 signatures, every other one of them a rejection the parser was designed to make. A person triaging this list has to read each one to find out that most of them are the target working correctly. On a real target, where rejections can happen at hundreds of places, the real crashes disappear entirely.
A fuzzer's value is its oracle: the fewer false findings it reports, the more each finding is worth. The check caught the change twice over: once because rejections appeared as crashes, and once because their saved inputs did not reproduce a crash at all when run again.
The rejection rule also changes what can be fuzzed. Many targets have no clean promise like this parser's, and a fuzzer for them needs a different oracle: a crash of the process, a sanitizer report, a timeout, or two implementations disagreeing about the same input. Choosing the oracle is the first design decision of any fuzzing campaign, and Module 2's crash triage depends on getting it right.
The opposite mistake is quieter and worse. An oracle that is too strict, one that ignored every exception type its author did not expect, would report this parser clean: the checksum bug raises struct's own error type, not one a hurried author would list.
A fuzzer that reports nothing has either found nothing or been told not to look, and the only way to tell which is to plant a bug you know about and confirm the fuzzer reports it. The check does exactly that for this exercise, and it is worth doing for every oracle you write.
What Random Changes Reach
Coverage, and the bug they missFinding one bug says nothing about what was missed. This script runs the draft for twenty thousand executions while recording which lines of the parser ran.
$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 cov.py
24 of 25 lines ran in 20,000 executions
never ran line 45 return f'{name} (priority {100 // int(name[5])})'
crashes: error at tlv.py:29
24 of 25 lines ran. The one that never did is inside the ADMIN branch, where the second planted bug lives: an administrator's name is ADMIN followed by a priority digit, the code divides by that digit, and a priority of zero divides by zero.
To get there, a mutation of the seeds has to produce the five bytes A, D, M, I, N in order at the start of a name record, followed by a zero. Each random change touches one byte and picks one value from 256, so that sequence is effectively out of reach. Five independent campaigns show the same thing.
$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 campaign.py
rng 1: error at tlv.py:29 first at 2
rng 2: error at tlv.py:29 first at 4
rng 3: error at tlv.py:29 first at 12
rng 4: error at tlv.py:29 first at 83
rng 5: error at tlv.py:29 first at 42
The checksum bug was found in every run, first hit between execution 2 and 83. The ADMIN bug was never found. This is the general shape of random mutation: shallow bugs, guarded by one byte, are found almost at once; bugs guarded by a magic value, a keyword or a checksum the input must get right are found almost never, however long the campaign runs.
Measuring which lines ran is called coverage, and it is the instrument that tells you what a campaign reached and what it did not.
Coverage-guided fuzzers use it as feedback as well: an input that runs a line no earlier input ran is kept and mutated further, so the fuzzer works its way into the program one new line at a time. The draft has no feedback, and the practice's stretch goal adds it.
Coverage has limits worth knowing before Module 2 leans on it. A line that ran is not a line that was tested: the checksum line ran thousands of times before the first input reached it with one byte instead of two.
Line coverage also says nothing about which values reached a line, which is what the ADMIN bug turns on. Coverage tells you where a campaign has not been, reliably; it tells you much less about whether the places it has been are safe.
One Good Seed
And the question Module 2 asksThe other way past a magic value is to start from an input that already has it. Here are the same five campaigns with one extra seed: a valid name record for ADMIN1, an administrator with priority one.
$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 campaign.py 5243090641444d494e31
rng 1: ZeroDivisionError at tlv.py:45 first at 361, error at tlv.py:29 first at 24
rng 2: ZeroDivisionError at tlv.py:45 first at 1039, error at tlv.py:29 first at 13
rng 3: ZeroDivisionError at tlv.py:45 first at 2, error at tlv.py:29 first at 4
rng 4: ZeroDivisionError at tlv.py:45 first at 2814, error at tlv.py:29 first at 72
rng 5: ZeroDivisionError at tlv.py:45 first at 1179, error at tlv.py:29 first at 4
The ADMIN bug was found in every run, first hit between execution 2 and 2814. The seed is valid input that the parser accepts, and nothing about it is a crash. One mutation turns it into one: flipping the lowest bit of the digit 1 makes it 0. The fuzzer did not get smarter. It was given a starting point near the bug.
Seeds come from several places in practice. A corpus of real files in the format gives the fuzzer structure that real software produces. Inputs that crashed the target before, kept as regression seeds, keep old bugs from returning unnoticed. The format's specification supplies keywords, field values and boundary cases a person would think to try, and that is the step that has always needed an expert to read the specification.
That is the idea Module 2 is built on. Seeds and dictionaries of keywords are where fuzzing knowledge goes: what the format looks like, what values matter, what an expert would try.
A model can propose them from a format's documentation or a protocol's specification, and Module 2 measures when model-proposed seeds beat random ones on the same target and when they do not. The measurement only means something if the fuzzer, the oracle and the coverage around it are sound, which is what this lesson built.
Read the first-hit numbers with care. Across the five runs the deep bug's first hit ranged from execution 2 to 2814, and the shallow bug's first hit moved too, because the extra seed changed which random choices went where.
Five runs show the pattern, every run found it within the budget, but not a rate. A claim that a seed makes a bug ten times faster to find needs many more runs per setting and the spread reported with it, which is exactly how Module 2 compares model-proposed seeds against random ones.
1# Session 0.2: a mutation fuzzer23Assistant: Claude Opus 5.5 (claude-opus-5-5), the SEC410 build session, 2026-11-1845## Held6- Four mutations, one per call; an empty input can only grow.7- ValueError is a rejection, skipped; anything else is a crash.8- Crashes keyed by exception type and innermost file and line.9- Deterministic for a given rng seed. check f02: PASS.1011## Measured12- Counting rejections as crashes: 5 signatures, one a bug.13- 20,000 executions: 24 of 25 lines ran; the ADMIN branch never.14- Five rng seeds: checksum bug first hit between execution15 2 and 83; ADMIN bug never.16- One seed containing ADMIN: found in all five runs, first17 between execution 2 and 2814.1819## For Module 220- No feedback: inputs that reached new lines are not kept.21- Grouping by innermost line merges different bugs that fail22 on the same line, and splits one bug that fails on two.
The review is the record a later campaign starts from. It names the target version, the seeds and the random seeds used, what was found, and what was not, so a person running the next campaign can tell whether a change found something new or only found the same thing again under different settings. Without it, two campaigns cannot be compared, and a fuzzer's results are anecdotes.
The review's last section names the two limits Module 2 starts from. The fuzzer keeps nothing it learns, and its crash grouping is a heuristic: two different bugs that fail on the same line are merged, and one bug that fails on two lines is split. Crash triage by root cause is a module's work, not a line's.
Practice
A mutation fuzzer the labBrief your assistant with the contract and the constraints, check its draft, then measure coverage and run the campaign with and without a seed you choose. Record the first hits and what your seed contained.
1# Practice 0.2: a mutation fuzzer23## Done when4- [ ] lab/foundations/fuzz.py meets its docstring's contract5- [ ] python -m lab check f02 prints PASS6- [ ] cov.py run: which lines never ran, and why7- [ ] campaign.py run with and without a seed of your own8 that reaches the ADMIN branch; first hits recorded9- [ ] records/0.2-<assistant>.md: brief, draft, runs, review1011## Constraints12- Fuzz only the lab's targets, which you are authorized to test.13- The fuzzer may not read the target's source for hints.1415## Stretch16- Keep any mutated input that runs a line no earlier input ran,17 and add it to the seeds. Measure the ADMIN bug again.
A fuzzer runs a target on generated inputs and reports the ones that fail in ways the target does not promise. Its oracle decides what counts, and a loose one buries real bugs among correct rejections.
Grouping by root cause turns many crashes into a few findings, and every finding needs an input that reproduces it. Random mutation finds shallow bugs fast and misses anything behind a magic value, which coverage makes visible and a good seed puts in reach. Fuzz only what you are authorized to test.