In this section

0.2 Fuzzing Fundamentals

Module 0

Introduction

Fuzzing finds bugs by running a program on a very large number of inputs nobody wrote by hand and watching for failures. It is one of the most productive ways to find memory corruption, parser bugs and crashes, it is how a great deal of vulnerability research is done, and Module 2 builds a fuzzer whose inputs a model helps choose.

Before a model can make a fuzzer better, you need to know what a fuzzer is made of, what it can and cannot reach, and how its results are judged.

This lesson builds a mutation fuzzer from nothing against a small parser with bugs planted in it. It starts with the target and the rule that decides what counts as a crash, has an assistant write the fuzzer to a contract, and runs it.

Then it measures three things: what happens when the crash rule is too loose, how much of the target twenty thousand executions actually reach, and what a single well-chosen seed does for a bug random changes never find. That last measurement is the question Module 2 is built on.

System: a fuzzer and its targetfuzzer: your codeSeedsknown-good inputsmutate()one byte changedCrash tablesignature, first input, count, first_attarget: the TLV parserparse(data)type, length, value recordsValueErrorrejected: fineOther exceptiona crashPlanted bugADMIN + a digit123data flowboundary1mutated input2outcome3crash signed by its innermost frame

The fuzzer owns the inputs and the record of what broke; the target owns the oracle. A ValueError is the parser refusing input as designed, so only other exceptions reach the crash table.

1

The Target and Its Promise

What counts as a bug

The target is a parser for a small record format, written for this lab. Its docstring is its specification.

Visual Studio Codetlv.py - sec410-lab
tlv.py
lab › foundations › targets › tlv.py
⋯ 5 lines folded6  0x07  checksum  a two-byte big-endian sum of every value byte before it, modulo 655367  0x09  name      a user name; an administrator's name is ADMIN and a priority digit from 1 to 989parse(data) returns the records as a list of (type, value). Input this format does not allow is rejected with ValueError:10that is the parser working, not a crash. Any other exception is a bug.11"""12import struct1314MAGIC = b'RC'151617def parse(data):18    if not isinstance(data, (bytes, bytearray)) or data[:2] != MAGIC:19        raise ValueError('not an RC stream: missing magic')20    out, i, total = [], 2, 021    while i < len(data):⋯ 28 lines folded
mainPythonUTF-8Ln 1, Col 1
The target: a record parser written for this lab, its rules stated in the docstring.

The most important line in the docstring is the one that says what a bug is. The parser promises to reject input it does not accept by raising ValueError. That is the parser working: a malformed record is supposed to be refused, and refusing it is correct.

Any other exception means the parser hit a state its author did not handle, which is a bug. A fuzzer needs a rule like this, usually called an oracle, that decides which outcomes are findings, and the rule comes from the target's specification, not from the fuzzer.

Try the rule by hand on four inputs: a valid stream, one with the wrong magic bytes, one with an unknown record type, and one with a checksum record that is one byte short.

$ python3 try.py
b'RC\x01\x05hello'     parsed    [(1, 'hello')]
b'XX\x01\x05hello'     rejected  not an RC stream: missing magic
b'RC\x05\x01!'         rejected  unknown record type 0x05 at offset 2
b'RC\x07\x01\x00'      CRASH     error: unpack requires a buffer of 2 bytes

The first parses. The second and third are rejected, with messages saying why, which is the promise being kept. Notice that the messages are specific: they say what was wrong and where. A rejection that only said invalid input would still keep the promise, and would make every rejection look the same to a person reading a fuzzer's output.

The fourth crashes. The checksum record is meant to carry two bytes, and the parser unpacks two bytes without first checking that two are there. That is a planted bug, and it is the kind fuzzing finds well: a length the code trusts and the input controls.

The crash's exception type reads simply as error, which is the name the struct module gives its own exception class.

It is easy to overlook in a list of crashes because it looks like a generic word rather than a type, and it is a good reminder that an oracle should be defined by what is allowed, the one rejection type the specification promises, rather than by a list of what is not. A list of forbidden exception types written from memory would almost certainly have missed this one.

2

Authorization First

What you may fuzz

The brief in section 4 says the target was written for this lab and that testing it is authorized. That sentence is not a formality.

A fuzzer sends a target inputs designed to make it fail, at a rate no person could, and pointed at a system you do not own or have written permission to test, it is an attack whatever the intent. Every target in this course is one the lab supplies, runs on your own machine, and exists to be tested.

The same discipline applies to how a fuzzer runs. A target that crashes may also hang, exhaust memory, or write files, so fuzzing runs in an environment where those failures are contained: a disposable directory, a process with limits, a machine whose state does not matter.

The lab's targets are pure functions with no side effects, which is why the exercise can run them in-process. Module 2's targets are larger, and its lessons run them the way a real campaign would. Module 12 returns to the same question from the other side: what you may do with a bug once you have found it.

Write the authorization down before the first execution, in a file that travels with the campaign. It names the target and its version, who owns it, who authorized the testing and when, where the fuzzer runs, and what happens to what it finds.

Visual Studio Codescope.md - sec410-lab
scope.md
scope.md
1# Fuzzing scope: SEC410 lab23Target      lab/foundations/targets/tlv.py, as shipped in the lab4Owner       the SEC410 course; written to be tested5Authorized  every student, on their own machine6Where       in-process, on your own machine, no network7Findings    stay in your records; there is no vendor to notify89Out of scope: any parser, service or file format you did not10receive in the lab pack, including ones that look the same.
mainMarkdownUTF-8Ln 1, Col 1
The authorization, written before the first execution.

The lab's scope file is short because the lab's situation is simple.

A real one is the document you point to when someone asks why a test environment fell over at three in the morning, or why a crash report reached the vendor before the press. It also constrains the assistant: an assistant asked to help fuzz a target should be given the scope file, and a target outside it is a request to decline.

3

The Contract

Two functions, stated before they are written

The exercise file states what the fuzzer must do.

Visual Studio Codefuzz.py - sec410-lab
fuzz.py
lab › foundations › fuzz.py
1"""Lesson 0.2: a mutation fuzzer. This file is the exercise: write mutate() and fuzz().23A fuzzer runs a target over and over on inputs it generates, and records the inputs that make the target fail in a way4it should not. A mutation fuzzer generates each input by changing a known-good one, a seed, slightly.56Contract, checked by `python -m lab check f02`:78    mutate(data, rng) -> bytes9        One random change to data: flip one bit, replace one byte, insert one byte or delete one byte, chosen with rng10        (a random.Random). Never returns the input unchanged unless it is empty and the change was a deletion.1112    fuzz(target, seeds, iterations, rng) -> {'executions': int, 'crashes': {signature: crash}}13        Run target(input) iterations times. Each input is mutate() applied to a seed chosen with rng. ValueError is the14        target's documented way of rejecting bad input: not a crash. Any other exception is a crash.15        signature   the root cause as a string: 'ExceptionType at file.py:line', the innermost frame of the traceback16        crash       {'input': the first input that hit it, as hex, 'count': how many inputs hit it,17                     'first_at': the execution number (from 1) of the first}1819The same seeds, iterations and rng seed must give the same result.20"""212223def mutate(data, rng):24    raise NotImplementedError('lesson 0.2: write mutate')252627def fuzz(target, seeds, iterations, rng):28    raise NotImplementedError('lesson 0.2: write fuzz')
mainPythonUTF-8Ln 27, Col 1
Two functions to write, and how their results are judged.

Two parts of the contract carry most of the weight. The first is the crash signature: a string naming the exception type and the file and line where it was raised. Twenty inputs that crash at the same line usually share one root cause, and a fuzzer that reported them as twenty crashes would bury the one fact that matters.

The second is determinism: the same seeds, iterations and random number seed must give the same result. A crash nobody can reproduce cannot be fixed, and a campaign whose result changes on every run cannot be compared with the next one.

Run the check against the file as shipped.

$ python3 -m lab check f02
f02  0.2 Fuzzing Fundamentals: lab/foundations/fuzz.py
FAIL  mutate() is written
      lesson 0.2: write mutate
FAIL

The check needs both functions and stops at the first one missing. Once they exist it tests each property separately, so a fuzzer that finds the bug but cannot reproduce it, or reproduces it but changes from run to run, is told exactly which property failed.

That matters more for fuzzers than for most code, because a fuzzer that looks like it works and does not is worse than none: it reports a target clean.

Before the brief, here is the procedure the contract describes, drawn as the steps and decisions the code has to make.

Procedure: a mutation fuzzerPick a seed with rngSame seeds and rng seed:the same result.Mutate: flip, replace,insert or delete one byteRun the target on itException other thanValueError?yesCrash: sign it by theinnermost framenoIterations left?yesNext inputnoReturn executions andcrashes by signatureOne entry per root cause,with its first input.stepdecisionnotestops or failsbranch ends

Each execution changes a seed by one byte and runs the target. A ValueError is the target refusing bad input as designed; any other exception is a crash, grouped by where it was raised.

Keep it open while reading the draft: each decision in the diagram should appear as a condition in the code, and each side branch as a case the check can test.

4

Session: The Brief

Write it blind to the target

The brief points at the contract and adds two constraints of its own.

SEC410 lab, lesson 0.22026-11-18
fuzz.pyPYtlv.pyPY

Write mutate() and fuzz() in fuzz.py to the contract in its docstring. The target is targets/tlv.py, a parser written for this lab with bugs planted in it, so we are authorized to test it; don't read its code for hints beyond its docstring, because the fuzzer should work on any target with the same contract. Standard library only.

fuzz.pyCode · PY · 62 linesDownload
Reply to Claude…Opus 5.5
Recorded unedited in assistant-records/m00/0.2-brief.md

The constraint that matters is the one telling the assistant not to read the target's code for hints. An assistant given the parser's source would notice the checksum record and the ADMIN branch, and could write a fuzzer that targets them directly.

That fuzzer would pass the check and be useless on the next target. A fuzzer is a general tool, and the brief keeps it one. The standard library constraint keeps the result small enough to read in full, for the same reason the agent loop in lesson 0.1 avoided a framework.

The blind constraint is also how fuzzing is done for real. Black-box fuzzing treats the target as something you can only run, not read; grey-box fuzzing adds feedback from the running target, such as which code each input reached; white-box approaches reason over the source. This lesson's fuzzer is black-box. Section 7 measures what that costs, and the practice's stretch goal turns it grey.

5

The Draft

Mutate, run, sort, group

The reply was the file. Here is the code below the docstring.

Visual Studio Codefuzz.py - sec410-lab
fuzz.py
lab › foundations › fuzz.py
21import os222324def mutate(data, rng):25    data = bytearray(data)26    ops = ['flip', 'replace', 'insert', 'delete'] if data else ['insert']27    op = rng.choice(ops)28    if op == 'insert':29        data.insert(rng.randrange(len(data) + 1), rng.randrange(256))30        return bytes(data)31    i = rng.randrange(len(data))32    if op == 'flip':33        data[i] ^= 1 << rng.randrange(8)34    elif op == 'replace':35        data[i] = rng.choice([b for b in range(256) if b != data[i]])36    else:37        del data[i]38    return bytes(data)394041def _signature(exc):42    tb = exc.__traceback__43    while tb.tb_next:44        tb = tb.tb_next45    return f"{type(exc).__name__} at {os.path.basename(tb.tb_frame.f_code.co_filename)}:{tb.tb_lineno}"464748def fuzz(target, seeds, iterations, rng):49    crashes = {}50    for n in range(1, iterations + 1):51        data = mutate(rng.choice(seeds), rng)52        try:53            target(data)54        except ValueError:55            continue56        except Exception as e:  # noqa: BLE001 - every other exception is a crash57            sig = _signature(e)58            if sig in crashes:59                crashes[sig]['count'] += 160            else:61                crashes[sig] = {'input': data.hex(), 'count': 1, 'first_at': n}62    return {'executions': iterations, 'crashes': crashes}
mainPythonUTF-8Ln 1, Col 1
The draft: four mutations, a rejection that is not a crash, and crashes grouped by where they happen.

The mutation function picks one of four operations at random and applies it once: flip a bit, replace a byte with a different value, insert a byte, delete one. An empty input can only grow, so its only operation is insert. The replace operation chooses from every value except the current one, which guarantees the input changes, as the contract requires.

The fuzzing loop picks a seed, mutates it, runs the target, and sorts the outcome. A ValueError is skipped. Anything else is a crash, and its signature is built from the innermost frame of the traceback, the line where the exception was actually raised.

The first input for each signature is kept, as hex so it can be stored and replayed; later inputs with the same signature only add to its count.

Two choices in the draft are worth checking against the contract rather than accepting. The signature uses the innermost frame, which for this parser is the line in its own code where the exception was raised; a target that called into a library would put the library's line there, and the signature would group crashes by where the library failed rather than where the target misused it.

And every mutation is applied to a seed, never to an earlier mutant, so each input is at most one change away from a known-good one. That keeps inputs close to valid, which suits a parser with strict structure, and it limits how far from the seeds the fuzzer can ever get.

Run the check.

$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 -m lab check f02
f02  0.2 Fuzzing Fundamentals: lab/foundations/fuzz.py
ok    mutate() returns bytes, one change at a time
      600 of 600 changed; length within one byte: True
ok    fuzz() counts its executions
      3000 executions
ok    the checksum bug is found
      error at tlv.py:29
ok    rejections (ValueError) are not crashes
      1 signatures
ok    every saved input reproduces its crash
      all reproduce
ok    the same rng seed gives the same result
ok    each crash records when it was first hit and how often
      error at tlv.py:29 after 2, hit 94 times
PASS

Every property holds. The check ran the fuzzer three thousand times against the parser, twice with the same random seed, and compared the results.

The checksum bug was found within the first handful of executions and hit dozens of times in three thousand, which is typical of a shallow bug: one byte in the right place triggers it, and random changes put bytes in the right place often. The saved input reproduces the crash when run again, and the same random seed gives the same result.

6

A Loose Oracle, Measured

Every exception a crash

The tempting simplification is to drop the rejection rule and treat every exception as a finding. More findings sounds like a better fuzzer. Here is that change.

Visual Studio Codefuzz.py (Working Tree) - sec410-lab
fuzz.py (Working Tree)
Source Control › Changes › lab/foundations/fuzz.py
53              target(data)-         except ValueError:-             continue54          except Exception as e:  # noqa: BLE001 - every other exception is a crash
main*1 file changed
The shortcut: treat every exception as a crash.

Run the check against it and read the crash list.

$ cp records/fuzz-noisy.py lab/foundations/fuzz.py
$ python3 -m lab check f02 | sed -n "6,10p"
ok    the checksum bug is found
      ValueError at tlv.py:19; ValueError at tlv.py:23; ValueError at
      tlv.py:31; ValueError at tlv.py:36; error at tlv.py:29
FAIL  rejections (ValueError) are not crashes
      5 signatures

The one real bug is still there, now among 5 signatures, every other one of them a rejection the parser was designed to make. A person triaging this list has to read each one to find out that most of them are the target working correctly. On a real target, where rejections can happen at hundreds of places, the real crashes disappear entirely.

A fuzzer's value is its oracle: the fewer false findings it reports, the more each finding is worth. The check caught the change twice over: once because rejections appeared as crashes, and once because their saved inputs did not reproduce a crash at all when run again.

The rejection rule also changes what can be fuzzed. Many targets have no clean promise like this parser's, and a fuzzer for them needs a different oracle: a crash of the process, a sanitizer report, a timeout, or two implementations disagreeing about the same input. Choosing the oracle is the first design decision of any fuzzing campaign, and Module 2's crash triage depends on getting it right.

The opposite mistake is quieter and worse. An oracle that is too strict, one that ignored every exception type its author did not expect, would report this parser clean: the checksum bug raises struct's own error type, not one a hurried author would list.

A fuzzer that reports nothing has either found nothing or been told not to look, and the only way to tell which is to plant a bug you know about and confirm the fuzzer reports it. The check does exactly that for this exercise, and it is worth doing for every oracle you write.

7

What Random Changes Reach

Coverage, and the bug they miss

Finding one bug says nothing about what was missed. This script runs the draft for twenty thousand executions while recording which lines of the parser ran.

$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 cov.py
24 of 25 lines ran in 20,000 executions
  never ran  line 45  return f'{name} (priority {100 // int(name[5])})'
crashes: error at tlv.py:29

24 of 25 lines ran. The one that never did is inside the ADMIN branch, where the second planted bug lives: an administrator's name is ADMIN followed by a priority digit, the code divides by that digit, and a priority of zero divides by zero.

To get there, a mutation of the seeds has to produce the five bytes A, D, M, I, N in order at the start of a name record, followed by a zero. Each random change touches one byte and picks one value from 256, so that sequence is effectively out of reach. Five independent campaigns show the same thing.

$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 campaign.py
rng 1: error at tlv.py:29 first at 2
rng 2: error at tlv.py:29 first at 4
rng 3: error at tlv.py:29 first at 12
rng 4: error at tlv.py:29 first at 83
rng 5: error at tlv.py:29 first at 42

The checksum bug was found in every run, first hit between execution 2 and 83. The ADMIN bug was never found. This is the general shape of random mutation: shallow bugs, guarded by one byte, are found almost at once; bugs guarded by a magic value, a keyword or a checksum the input must get right are found almost never, however long the campaign runs.

Measuring which lines ran is called coverage, and it is the instrument that tells you what a campaign reached and what it did not.

Coverage-guided fuzzers use it as feedback as well: an input that runs a line no earlier input ran is kept and mutated further, so the fuzzer works its way into the program one new line at a time. The draft has no feedback, and the practice's stretch goal adds it.

Coverage has limits worth knowing before Module 2 leans on it. A line that ran is not a line that was tested: the checksum line ran thousands of times before the first input reached it with one byte instead of two.

Line coverage also says nothing about which values reached a line, which is what the ADMIN bug turns on. Coverage tells you where a campaign has not been, reliably; it tells you much less about whether the places it has been are safe.

8

One Good Seed

And the question Module 2 asks

The other way past a magic value is to start from an input that already has it. Here are the same five campaigns with one extra seed: a valid name record for ADMIN1, an administrator with priority one.

$ cp records/fuzz-0.2.py lab/foundations/fuzz.py
$ python3 campaign.py 5243090641444d494e31
rng 1: ZeroDivisionError at tlv.py:45 first at 361, error at tlv.py:29 first at 24
rng 2: ZeroDivisionError at tlv.py:45 first at 1039, error at tlv.py:29 first at 13
rng 3: ZeroDivisionError at tlv.py:45 first at 2, error at tlv.py:29 first at 4
rng 4: ZeroDivisionError at tlv.py:45 first at 2814, error at tlv.py:29 first at 72
rng 5: ZeroDivisionError at tlv.py:45 first at 1179, error at tlv.py:29 first at 4

The ADMIN bug was found in every run, first hit between execution 2 and 2814. The seed is valid input that the parser accepts, and nothing about it is a crash. One mutation turns it into one: flipping the lowest bit of the digit 1 makes it 0. The fuzzer did not get smarter. It was given a starting point near the bug.

Seeds come from several places in practice. A corpus of real files in the format gives the fuzzer structure that real software produces. Inputs that crashed the target before, kept as regression seeds, keep old bugs from returning unnoticed. The format's specification supplies keywords, field values and boundary cases a person would think to try, and that is the step that has always needed an expert to read the specification.

That is the idea Module 2 is built on. Seeds and dictionaries of keywords are where fuzzing knowledge goes: what the format looks like, what values matter, what an expert would try.

A model can propose them from a format's documentation or a protocol's specification, and Module 2 measures when model-proposed seeds beat random ones on the same target and when they do not. The measurement only means something if the fuzzer, the oracle and the coverage around it are sound, which is what this lesson built.

Read the first-hit numbers with care. Across the five runs the deep bug's first hit ranged from execution 2 to 2814, and the shallow bug's first hit moved too, because the extra seed changed which random choices went where.

Five runs show the pattern, every run found it within the budget, but not a rate. A claim that a seed makes a bug ten times faster to find needs many more runs per setting and the spread reported with it, which is exactly how Module 2 compares model-proposed seeds against random ones.

Visual Studio Code0.2-session.md - sec410-lab
0.2-session.md
records › 0.2-session.md
1# Session 0.2: a mutation fuzzer23Assistant: Claude Opus 5.5 (claude-opus-5-5), the SEC410 build session, 2026-11-1845## Held6- Four mutations, one per call; an empty input can only grow.7- ValueError is a rejection, skipped; anything else is a crash.8- Crashes keyed by exception type and innermost file and line.9- Deterministic for a given rng seed. check f02: PASS.1011## Measured12- Counting rejections as crashes: 5 signatures, one a bug.13- 20,000 executions: 24 of 25 lines ran; the ADMIN branch never.14- Five rng seeds: checksum bug first hit between execution15  2 and 83; ADMIN bug never.16- One seed containing ADMIN: found in all five runs, first17  between execution 2 and 2814.1819## For Module 220- No feedback: inputs that reached new lines are not kept.21- Grouping by innermost line merges different bugs that fail22  on the same line, and splits one bug that fails on two.
mainMarkdownUTF-8Ln 11, Col 1

The review is the record a later campaign starts from. It names the target version, the seeds and the random seeds used, what was found, and what was not, so a person running the next campaign can tell whether a change found something new or only found the same thing again under different settings. Without it, two campaigns cannot be compared, and a fuzzer's results are anecdotes.

The review's last section names the two limits Module 2 starts from. The fuzzer keeps nothing it learns, and its crash grouping is a heuristic: two different bugs that fail on the same line are merged, and one bug that fails on two lines is split. Crash triage by root cause is a module's work, not a line's.

Do

Practice

A mutation fuzzer
the lab

Brief your assistant with the contract and the constraints, check its draft, then measure coverage and run the campaign with and without a seed you choose. Record the first hits and what your seed contained.

Visual Studio Code0.2.md - sec410-lab
0.2.md
practice › 0.2.md
1# Practice 0.2: a mutation fuzzer23## Done when4- [ ] lab/foundations/fuzz.py meets its docstring's contract5- [ ] python -m lab check f02 prints PASS6- [ ] cov.py run: which lines never ran, and why7- [ ] campaign.py run with and without a seed of your own8      that reaches the ADMIN branch; first hits recorded9- [ ] records/0.2-<assistant>.md: brief, draft, runs, review1011## Constraints12- Fuzz only the lab's targets, which you are authorized to test.13- The fuzzer may not read the target's source for hints.1415## Stretch16- Keep any mutated input that runs a line no earlier input ran,17  and add it to the seeds. Measure the ADMIN bug again.
mainMarkdownUTF-8Ln 3, Col 1

A fuzzer runs a target on generated inputs and reports the ones that fail in ways the target does not promise. Its oracle decides what counts, and a loose one buries real bugs among correct rejections.

Grouping by root cause turns many crashes into a few findings, and every finding needs an input that reproduces it. Random mutation finds shallow bugs fast and misses anything behind a magic value, which coverage makes visible and a good seed puts in reach. Fuzz only what you are authorized to test.