The Hunt That Found Nothing

An advisory is published on a Tuesday. It names a technique, gives three indicators, and your CISO asks the question every CISO asks: are we affected.

You do the right thing. You translate the indicators into data classes, pick a cheap artifact, rehearse it against a handful of hosts, and launch it across the estate. By Wednesday afternoon it has reached most of the fleet and returned nothing at all. You write "the indicators were not observed on our estate" and the matter closes.

The tool worked perfectly. The sweep was well scoped and well run. And the sentence is wrong in a way nothing on your screen will tell you.

A fleet result has a numerator and no denominator

Whatever platform you hunt with, the mechanism is the same. Machines carrying an agent check in, receive the question, answer it, and the count of answers goes up. What the platform reports is how many responded and what they said.

What it cannot report is how many exist.

Your server knows how many clients have ever enrolled. It does not know how many machines your organization owns, how many of those were rebuilt without the agent, how many were provisioned by a process nobody updated, or how many are sitting in a drawer. That number lives in an asset inventory, a directory, a device management platform, or in nobody's system at all, and no amount of querying endpoints will produce it.

So a clean result supports a narrow claim: this was not found on the machines that answered, during the window you read it. Anything wider is an assertion built on information the tool does not hold.

The gap is usually larger than people expect, and it is not random.

The query that gives you the denominator

Whatever you hunt with, the reconciliation is the same shape: what the security tool believes it reaches, against what a source of truth outside it says exists.

In Defender XDR the security half is one query. DeviceInfo carries both an onboarding state and a sensor health state, and the two disagree more often than people expect.

// Coverage, as the security tool sees it. Run before you report a clean sweep.
// OnboardingStatus is what was INTENDED. SensorHealthState is what is TRUE.
// A device can be "Onboarded" and have sent nothing for weeks.
DeviceInfo
| where Timestamp > ago(30d)
| summarize arg_max(Timestamp, OnboardingStatus, SensorHealthState, OSPlatform)
    by DeviceId, DeviceName
| summarize Devices = count(),
            LastSeen = max(Timestamp)
    by OnboardingStatus, SensorHealthState
| order by Devices desc

The rows worth reading are the ones that are not Onboarded / Active. Inactive means the sensor has not reported for a while, and those devices are counted in your console total while contributing nothing to any hunt. Misconfigured is worse, because it looks onboarded from every summary view and reports partial telemetry.

Then the same question asked of the machines that answered your specific sweep, rather than of the estate in general:

// Which devices actually contributed to the hunt window, and which did not.
// Change the table to whichever one your hunt queried.
let HuntWindow = ago(2d);
let Answered = DeviceProcessEvents
    | where Timestamp > HuntWindow
    | distinct DeviceId;
DeviceInfo
| where Timestamp > ago(30d)
| summarize arg_max(Timestamp, DeviceName, OSPlatform, SensorHealthState) by DeviceId
| extend Contributed = iff(DeviceId in (Answered), "answered", "SILENT")
| summarize Devices = count() by Contributed, SensorHealthState, OSPlatform
| order by Contributed asc, Devices desc

The SILENT rows are your actual gap for that hunt. Grouping them by platform and sensor state usually splits them into three populations: sensors that are genuinely unhealthy, machines that were simply off, and a platform you forgot was in scope.

The non-responders are not a random sample

This is the part that turns an academic point into an operational one.

If the machines that did not answer were a random 10 percent of the estate, a clean result across the other 90 percent would be reasonably strong evidence. They are not random. The population that fails to answer a hunt is systematically different from the population that answers it, and the difference runs in exactly the wrong direction.

Laptops that were closed. Machines belonging to people on leave, traveling, or working from a home network that day. Hosts in a site with an outage. Kit issued to contractors. Machines that were rebuilt last month by somebody who did not know the agent was part of the build.

Look at that list and ask which of those you would most expect an intrusion to be sitting on. It is not the always-on desktop in the office that answered in four seconds.

There is a second population that never appears in the count at all, because it was never enrolled. Acquired companies still on their own tooling. The manufacturing segment nobody wanted to touch. Personal devices in a bring-your-own arrangement. Servers built before the deployment standard existed. Those machines are not in the 812, so they cannot be in the 731, and a hunt reaching every single enrolled client still says nothing whatsoever about them.

What the numbers looked like on a real sweep

Take a mid-sized estate. The console reports 812 enrolled clients. You scope the hunt to Windows, excluding a handful of machines under legal hold and a few fragile production systems, and 694 fall inside the scope.

The hunt runs. By the time you read it, 661 have completed and 7 have returned errors, which turns out to be a path that does not exist on an older build shared by all seven. So 654 machines actually answered the question.

Now reconcile against the source of truth outside the platform. The asset inventory lists 866 endpoints. Comparing the two lists produces about 40 machines with no agent at all, and, more interestingly, a handful with an agent and no inventory entry.

The finding you can defend reads like this:

// The finding, with its population attached. Five sentences, not one.
NOT OBSERVED  on 654 of 812 known clients, checked 8-9 April.
DID NOT ANSWER  51 in-scope clients in that window; 7 returned errors.
GAP SHAPE      the non-responding set is disproportionately laptops.
NOT COVERED    ~40 machines with no agent, per the inventory comparison.
STILL RUNNING  hunt open until 15 April; late machines collect on return.

That is five sentences instead of one. It is also the version that survives somebody asking a second question, and it is the version that tells your CISO something true about the organization's actual exposure rather than about your tooling's reach.

The three checks that make a clean result defensible

None of these take long. All three are skipped routinely, because a result with no hits does not feel like it needs interrogating.

Count the states before you count the rows. A fleet result is a mixture of collections that completed, collections that were canceled at a resource limit, collections that errored partway, and hosts that were skipped by a precondition. Those all look identical in an empty result set, and only one of them means "this machine does not have it".

On an agent platform that reports per-host collection state, this is one query, and it is the one that decides whether the rest of the analysis is safe:

-- Velociraptor: the states behind a hunt, before you read a single row.
-- FINISHED counts collections that STOPPED, not collections that SUCCEEDED,
--   so a host that errored is "completed" as far as any summary is concerned.
SELECT state AS State, count() AS Hosts
FROM hunt_flows(hunt_id="H.2026.04.08.A")
GROUP BY State
ORDER BY Hosts DESC

State       Hosts
FINISHED      654      <- the only rows your finding actually covers
ERROR           7      <- a path missing on an older build; never asked

Look at the non-responders as a list, not a number. Fifty machines that did not answer is a rounding error if they are all decommissioned desktops awaiting collection. It is a serious gap if eleven of them belong to the department the advisory's technique targets, or if a third are laptops from the site where your last incident started. That distinction takes about a minute and it is the difference between a gap and a blind spot.

Reconcile the client list against something outside the platform, in both directions. Machines in the inventory with no agent are the gap everybody expects. Machines with an agent and no inventory entry are the more interesting finding, because an endpoint your organization does not know it owns is a question for somebody outside security, and occasionally the start of one for you.

If your inventory is reachable as a watchlist or a lookup table, the comparison is one join and worth scheduling monthly rather than running once:

// Both directions. The second half is the one nobody runs.
let Inventory = _GetWatchlist("AssetInventory") | project AssetName = tostring(Column1);
let Agents = DeviceInfo
    | where Timestamp > ago(30d)
    | summarize arg_max(Timestamp, SensorHealthState) by DeviceName;
// In the inventory, no agent: the expected gap
Inventory | join kind=leftanti (Agents) on $left.AssetName == $right.DeviceName
| extend Finding = "in inventory, NO AGENT"
| union (
// Has an agent, not in the inventory: the interesting one
Agents | join kind=leftanti (Inventory) on $left.DeviceName == $right.AssetName
| extend Finding = "AGENT, not in inventory")
| summarize Devices = count() by Finding

Why this is worse than a broken control

A control that has quietly stopped working is a familiar failure and there are ways to catch it. Volume drops, a dashboard goes flat, somebody eventually notices the silence.

This is a different shape and it is harder. Nothing failed. The agent worked. The query was correct. The hunt ran, reported honestly, and returned exactly what it found. Every component behaved as designed, and the defect is entirely in the sentence a human wrote afterwards.

That means no monitoring catches it, no health check surfaces it, and no error appears anywhere. The only thing standing between a correct tool and an incorrect conclusion is whether somebody attached the population to the claim.

It also means the failure compounds quietly. A negative finding closes a matter. Nobody revisits it, nobody re-runs it, and the machine that was in a drawer during the sweep comes back online the following week carrying the thing you told the board you did not have.

The same arithmetic outside endpoint hunting

This is not a Velociraptor problem, or an EDR problem, or a hunting problem. It is a property of any answer assembled from things that report in.

A vulnerability scan covers the hosts that responded to the scanner during the window, not the hosts on the network. A configuration audit covers the machines the management platform currently manages. A log-based hunt covers the sources that were shipping logs at the time, which is not the same as the sources that exist, and a source that silently stopped forwarding three weeks ago produces exactly the same clean result as a source with nothing to report.

In every case the tool reports its numerator accurately and the denominator is somebody else's data. In every case the failure mode is a human writing a sentence one scope wider than the evidence.

The habit that fixes all of them is the same: put the population in the sentence, not in a footnote. It costs one clause, it reads as more careful rather than less confident, and it answers the reviewer's next three questions before they are asked.

What to do this week

Take your last clean fleet-wide finding and reread the sentence you wrote. Does it name a number of machines, or does it say "the estate"? If it says the estate, you claimed a denominator your tooling does not hold.

Run the two-direction reconciliation above. Under an hour on most estates. The machines with an agent and no inventory entry are the finding worth taking to somebody outside security.

On your next hunt, run the state query before you read a single row. If more than one collection in twenty is anything other than completed, restrict the claim to the completed set and say why in the finding.

Pull the list of machines that did not answer your last sweep and look at what they are. Not how many. What. If they cluster by site, department or device type, that is a blind spot with a shape, and the shape usually matters.

Add the population clause to your reporting template. "Not observed on N of M known clients between these dates" instead of "not present". Once it is in the template nobody has to remember it.

Take this further

The reconciliation is the part worth building once and keeping. Whatever holds your source of truth, whether that is an asset inventory, a directory, or a device management platform, a monthly comparison against your agent list is the only mechanism that turns a coverage figure into a coverage claim. Everything else on this list is a habit; this one is a process with an owner.

The broader lesson generalizes past coverage. Any time a system reports what it reached rather than what exists, the reported unit and the useful unit have come apart, and the gap between them is invisible in the output. A hunt reports responders. A scan reports responders. A survey reports respondents. When somebody hands you a clean result from any of them, the question worth asking is not what it found but what it covered, and the answer is almost never on the same screen as the finding.

Our Velociraptor for Endpoint Investigation course works this in detail, including the collection states that make a partial result look complete and the prevalence analysis that ranks a malicious entry as ordinary. The first module is free and needs no account.