0.1 What a YARA Rule Is For

The claim a rule makes

Every rule you write is a sentence about files that do not exist yet.

rule Loader_X { ... } does not say "this sample is a loader." The sample is already identified; you have it open. What the rule says is: any file matching this description is the same thing as my sample, and it says that about the entire population of files it will ever be run against, most of which nobody has written yet.

That is a large claim from one example, and every difficulty in this course follows from it.

Compare it to the thing it is most often confused with. A hash lookup is a claim about one file: this exact sequence of bytes is known bad. It is precise, it is trivially verifiable, and it is worthless the moment anybody recompiles, because the claim was only ever about the one artifact. A YARA rule is the opposite trade. It gives up the certainty in exchange for reaching files the hash never could, and everything you do while writing one is managing how much certainty you give up for how much reach.

That framing decides what a mistake even looks like. If you think of a rule as a description of your sample, then the rule is correct when it matches your sample, and you are finished in about four minutes. If you think of it as a hypothesis about a population, the rule is correct when it matches the right members of a population you cannot see, and you are not finished until you have found some way to test that.

Here is the thing that makes it concrete. Write a rule from a sample, run it against that sample, and it fires. That result tells you nothing at all, and it feels like success.

# The test that cannot fail
yr scan myrule.yar sample.bin
#
# The two that can
yr scan myrule.yar ./siblings/        # does it generalize?
yr scan -r myrule.yar /usr/bin/       # does it over-reach?

Three commands. The first is the one everybody runs and the only one that proves nothing. It is the single most common mistake in YARA work, and it is why Section 5.2 spends its time on a corpus of files the rule was not built from.

Think about why the result is empty. You extracted the patterns from that file. Of course they are in that file. The test has no way to fail, and a test that cannot fail tells you nothing when it passes. The only informative tests are against files the rule did not come from: siblings, to see whether it generalizes, and clean software, to see whether it over-reaches.

Section 0.6 has you build a family of three near-identical samples for exactly this reason. Without siblings there is no way to demonstrate that a rule generalizes, and a course that never demonstrates it is teaching you to write rules you cannot evaluate.

The two ways a rule fails

They are not symmetrical, and treating them as though they are is how estates end up with detection sets nobody trusts.

It misses. The actor recompiles, changes a string, rotates a domain, and the rule stops firing. Nothing announces this. The rule sits in the deployment set returning zero, looking exactly like a rule protecting a clean estate. You find out during the next incident, or you never find out.

That last clause deserves reading twice. A rule that has silently stopped working and a rule protecting an estate where nothing bad is happening produce identical output: nothing. No error, no warning, no degraded state, just a rule in a repository returning zero hits the way it did yesterday. There is no signal to notice.

It fires too often. The rule matches legitimate software. This one announces itself immediately, loudly, and to people whose time you are wasting. Forty hits a day for a week and somebody disables it, usually without telling you, and now you have a rule in the repository that is not running.

The economics here are worth being precise about, because "false positives are bad" is not a useful principle on its own. A rule producing one false positive a month across a large estate is fine, and a rule producing three a day is not, and the difference is not the rate but what the rate does to the people receiving it. Analysts learn very quickly which rules waste their time, and once a rule has that reputation its true positives get dismissed along with everything else. A noisy rule does not just cost the time it wastes: it degrades the response to every hit it ever produces.

Most rule-writing advice optimizes against the second failure, because it is the one people experience. This course weights the first one more heavily, because a detection set full of quietly dead rules is worse than having no detection set: it produces confidence without coverage.

Both at once, which is the usual case

The two are not alternatives. A rule keyed on a compiler artifact and a common API string will do both simultaneously: miss the actor's next build because the compiler changed, and fire on unrelated software because the API string is everywhere.

That is what an untested rule usually looks like. Not obviously bad, just keyed on the wrong things, failing in both directions at once, and nobody knows because nobody measured it against anything.

It is worth naming why this is the default outcome rather than an unlucky one. The patterns that jump out of a sample when you read it are the ones that are visually distinctive: a long string, an unusual path, a recognizable API name. Visual distinctiveness has almost no relationship to either durability or rarity, and it is the property the human eye selects for. Left to instinct, you will reliably pick the wrong patterns, which is why Module 2 gives you a procedure instead.

What a good rule is keyed on

The short answer, which Module 2 spends six sections earning: things that cost the author something to change.

A file hash costs nothing, often not even a decision. A string in the binary costs a recompile. A mutex name costs a code edit and a rebuild, and the author has to accept that their own duplicate-instance check breaks if they get it wrong. A structural pattern in the code costs a rewrite of the routine.

Those are not equally good indicators and they are not equally specific either. The most durable ones are frequently the least distinctive, which is the tension Module 2 is about.

The direction of that tension catches people out. It feels as though the indicator that survives longest should also be the better one, and frequently it is the reverse. Behavioral and structural patterns survive rebuilds precisely because they describe something the program has to do, and things a program has to do are things many programs do. Meanwhile the most distinctive string in the sample, the one that looks like a perfect indicator, is often distinctive because it was an accident, and accidents do not survive the next build.

The other half: what your scanner can do with it

A rule is also something somebody has to execute, usually across a lot of files at once.

A rule that is correct and takes four seconds per file is not deployable across a fleet. A rule that needs a module your scanning surface does not load will not compile there. A rule that uses YARA-X syntax will not compile at all on a THOR deployment running 4.x, and you will discover that at deployment time rather than at authoring time unless you know to check.

That is why the deployment surface appears this early, in Section 0.4, rather than being left to the end. What counts as a good rule depends on where it runs, and deciding that after writing it is the wrong order.

The same rule can be excellent on one surface and unusable on another. Scanning a single forensic image, you can afford a rule that takes a second per file and uses every module available. Sweeping ten thousand endpoints during an active incident, you cannot, and the constraint is not really the CPU: it is that a slow fleet-wide scan produces its answer after the decision it was supposed to inform has already been made.

What this section establishes

A rule is a hypothesis about files you have not seen, generalized from one you have. It fails silently by missing and loudly by over-firing, and the silent failure is the one that costs you. It is keyed well when it is keyed on things the author would find expensive to change, and it is deployable when the surface it runs on can actually execute it.

Everything in the next seven modules is one of those four sentences worked out in detail.

Section 0.2 lays out how they are distributed across the modules, and why one of them did not exist eighteen months ago.