0.4 Where Your Rules Will Run

A rule is never deployed in the abstract. It runs somewhere specific, against a particular kind of target, under a time budget somebody else set, on an engine you probably did not choose. All four of those change what counts as a good rule, and the differences are large enough that a rule tuned for one surface can be unusable on another. This section covers the four places rules actually run, the yr scan options that exist because of them, and the reason deciding the surface after writing the rule is the wrong order.

The four surfaces

A forensic image is the most permissive. You have a mounted volume, nobody is waiting minute by minute, and you can install whichever engine you like and load every module. It is also the only surface you can rescan: the evidence is static and still there tomorrow, so a first pass that misses costs nothing but the time. That makes it the right place to experiment with a rule you are unsure about.

The trap on an image is different from the others and it is volume. A disk holds hundreds of thousands of files, so a rule producing a hit on one in a thousand produces hundreds of hits. That is not a false-positive problem in the ordinary sense, since a human may agree each hit was reasonable; it is a triage problem, and it is why a rule that is acceptable on a single suspect file can be unusable across a whole volume.

An endpoint sweep runs against a live machine, usually through an agent, in minutes rather than hours because a user is working on it. Two constraints appear here that the image does not have. Files change underneath the scan and some are locked, so a scan that cannot open a file typically skips it and reports on what it did read. And the engine is whatever the agent ships, which you did not choose and cannot change.

The locked-file behavior is the one worth dwelling on, because it is silent in the way Section 0.1 described. A file the scanner cannot open is skipped and the scan reports on everything it could read, so the number of files examined and the number of files present are different numbers and only one of them appears anywhere. On a live Windows host that gap is not small: the files most likely to be locked are the ones currently executing.

A fleet hunt asks one question of thousands of machines and needs the answer while it still matters. The constraint is not really CPU: it is that a scan finishing after the containment decision has been made is not an answer at all. Fleet rules are therefore narrow, fast, and few. You do not run a whole rule set across ten thousand endpoints; you run the two rules that answer today's question.

Narrowness buys something beyond speed, and it is the part people miss. A hunt returning four hits across ten thousand hosts is a result somebody can act on that afternoon. The same hunt returning nine hundred hits is not a result at all: it is a second triage problem, and the rule that produced it has converted one question into a larger one.

A mail or upload gateway runs inline in the delivery path, with a budget measured in fractions of a second, on an engine chosen by a vendor. Realistically you may not be able to deploy your own rules there at all. Where you can, the precision bar is much higher, because a false positive is not an analyst losing ten minutes but a blocked invoice with somebody's name attached to it.

It is also the surface where a rule is worth the most when it works, which is why it is worth the higher bar. Catching something before delivery prevents the incident rather than finding it afterwards, and the value of that is difficult to overstate next to the same rule finding the same file on a disk image three weeks later.

What the surface changes

Three things, and each has a corresponding option in the tool.

How much you can afford per file. An image tolerates an expensive rule; a fleet does not. Module 6 measures rule cost properly, and --timeout bounds it in the meantime.

Worth noting that cost is a property of the rule rather than of the surface, so the same rule is expensive everywhere. What changes is whether you can afford it, and a rule developed on an image without any attention to cost will be discovered to be unaffordable at the point somebody tries to run it across a fleet, which is the worst moment to find out.

How many files are in scope. Scanning every file on every endpoint is almost never what you want, and restricting the target removes far more work than optimizing the rule ever will.

The arithmetic makes the point better than the principle does. A typical endpoint holds a few hundred thousand files, and a fleet of ten thousand endpoints is therefore a few billion file scans. Halving the cost of a rule halves a very large number and it is still a very large number; scanning only user-writable directories, or only files modified in the last thirty days, removes most of the population before the rule is involved at all. Scope first, optimize second, and the ordering is not close.

Which engine has to compile it. This is the one people discover last and it is Module 7's subject. Most deployed scanners still embed YARA 4.x, so the rule that compiles cleanly on your laptop may not load on the thing that has to run it, and on some platforms that failure is silent: the rule is skipped while the rest of the set loads, leaving a deployed set quietly missing the rule you most wanted in it.

Scoping the target

The most useful option for fleet work is the one that stops you scanning a whole filesystem. --scan-list takes a file of paths rather than a directory, which is how you scan a set somebody else determined: the files an EDR flagged, the contents of a temp directory, everything modified in the last thirty days.

printf '/usr/bin/curl\n/usr/bin/bash\n/usr/bin/ls\n' > targets.txt
yr scan --scan-list rule.yar targets.txt
S /usr/bin/curl
S /usr/bin/bash
S /usr/bin/ls

That is a different operation from pointing at a directory, and it is the shape most real hunts take. The list is produced by something else that knows the estate, and YARA answers a question about exactly those files.

It also changes who owns the scoping decision, which is worth being conscious of. When you point at a directory you decided what to scan; when you take a list you inherited somebody else's decision, and any gap in their list is a gap in your coverage that will not appear anywhere in your results. That is Section 0.1's silence in an organizational form, and the answer is the same: know what the denominator was before you report on the numerator.

Bounding the work

Two options bound a scan that would otherwise run away, and one of them has a syntax trap worth meeting here rather than during an incident.

--skip-larger takes a size in bytes, and only bytes. A human-looking value is rejected outright rather than interpreted:

yr scan -z 100KB -c rule.yar /usr/bin/
error: invalid value '100KB' for '--skip-larger <FILE_SIZE>': invalid digit found in string

It fails loudly, which is the good case. Write -z 100000 and the scan runs, skipping anything above that size. Whether skipping large files is wise depends entirely on what you are hunting, and it is a decision rather than an optimization: an installer carrying a payload is a large file, and a size cap set for speed can exclude exactly the thing you are looking for.

--timeout bounds the time spent per file, which matters most when a rule contains something pathological. --threads sets how many workers run, defaulting to the core count, and on a live host that default is frequently wrong: using every core to scan competes with whatever the machine is supposed to be doing.

A per-file timeout has a consequence worth thinking through before you set one. A file that times out is not scanned, and it does not appear in the output as a failure any more than a file that was scanned and did not match. So a timeout converts a slow rule into a silent coverage gap, which is the same shape as every other problem in Section 0.1. If you set one, you want a record of what hit it, and Module 6 covers getting that.

Writing for a surface

Three habits follow, and they run through the rest of the course.

Know the target before you write. The surface sets the time budget, the engine and the acceptable false-positive rate. Writing first and asking later means discovering the constraint after the work.

In practice you frequently know the surface before you know anything else, because the surface is what generated the request. Somebody handing you a sample from a disk image wants a rule for that image; somebody asking whether a threat is present across the estate wants a fleet rule. Listening to which question was asked tells you most of what the constraint will be, before you have opened the sample.

Write to the most restrictive engine you will deploy to. If any of your surfaces runs 4.x, then your rules are 4.x-compatible rules and YARA-X is your authoring environment rather than your target. Module 7 covers doing that deliberately instead of by accident.

The alternative, maintaining two versions of a rule for two engines, sounds reasonable and goes wrong quickly. The two copies drift, a fix applied to one is forgotten on the other, and the version deployed to the surface you look at least is the one that rots. One rule written to the lower common denominator is almost always better than two written to their respective ceilings.

Assume the rule outlives the incident. The rule you write today for one fleet hunt ends up in the standing set, running on surfaces you were not thinking about. That is the argument for the metadata Module 1 covers: a rule that records what it was for and what will defeat it survives the transition; one that does not becomes somebody else's problem.

There is a lever that makes the transition easier and Module 7 covers it properly: the same rule can be deployed with a different threshold on different surfaces. A quantified condition requiring two patterns on an image and four on a gateway is one rule expressed twice, which is frequently a better answer than maintaining two rules that drift apart.

Surface guide: run the same rule three ways

Step 1, scan a directory with your Section 0.2 family and note the time it takes and the output shape.

Step 2, build a scan list naming three specific files and run the same rule with --scan-list. Note that the output is identical in form and the operation is completely different: you asked about named files rather than about a location.

Step 3, add a size bound with -z and a plain byte value. Try a human-readable value first so you meet the error deliberately rather than during a hunt.

Step 4, bound the threads with -p 2 and consider what that number should be on a machine somebody is using.

Step 5, write down which surface your rules are actually for, before you write another one. If the answer is more than one, note which is the most restrictive.

Section 0.5 covers what a match actually establishes once it fires, which turns out to be considerably less than the word "match" suggests.