What an Identity Detection Consists Of

Introduction

The word detection has been doing a lot of work in the last two sections without being defined. You will finish this one able to name the five parts a detection is made of, say which of them fail without producing an error, and explain why the query is the part least likely to be wrong.

Almost everything written about detection engineering is about queries. Rule repositories are collections of queries. Detection-as-code pipelines version queries. Conference talks show queries. The implicit model is that the difficulty of detection is expressing the malicious behavior precisely, and that once you have done so you have a detection.

The scenario at the top is what that model misses. The query was correct on the first attempt, has been correct for eighteen months, and has produced three true positives, which by any measure of query quality makes it an excellent rule. It has also never resulted in anybody doing anything, because of a dropdown nobody thought about at creation time.

A detection is five things and the query is one of them. The others are the data it depends on, the threshold or condition that decides what counts, the process it joins against once it fires, and the person who reads the result. All five have to work. Four of them fail without producing an error anywhere.

That asymmetry is the reason this course spends more time on the other four than on the first. A query that is wrong announces itself: it does not parse, or it returns something obviously absurd, or a colleague reviewing it says so. A severity set to Informational in a queue filtered to Medium produces exactly the same green dashboard as a well-routed rule, forever.

It is worth being fair to the query-centric view. A query can be shared between organizations, versioned, tested against sample data and reviewed by somebody who has never seen your tenant. Everything else is local: your severity scheme, your queue, your analysts, your data. The industry talks about queries because queries are the part that travels.

The cost is a body of published material describing one fifth of the problem, and estates assembled by importing other people's queries into local conditions nobody adjusted.

The rest of this section takes the five parts one at a time. Three of them get a module of their own later in the course, so the aim here is recognition rather than depth: enough to look at a rule somebody else wrote and ask which of its five parts has never been examined. In most estates the answer is four of them, and in the estate from section 0.1 it was four of them across all forty-seven identity rules.

Laying the five out in the order they execute makes the pattern visible, and the pattern is that failure gets quieter as you move right.

Take them in order. Data is whether the table the rule queries is present and populated, which section 0.2 covered and which is decided outside the rule entirely. Query is the expression of the behavior, and it is the only stage that fails loudly. Threshold is the condition deciding what counts as enough: how many failures, over what window, from how many addresses. Routing is everything between the rule producing a row and a human seeing it, which is severity, incident grouping, queue filters and ownership. Reader is whether the person who sees it does anything, which section 0.1 showed can be defeated by the rule's own neighbors.

Read the row of labels under the chevrons rather than the chevrons themselves. Data fails silently, as section 0.2 established, because a query against a table nothing writes to returns an empty set. Threshold fails silently, because a condition set too high produces no alerts and a condition set too low produces alerts nobody reads, and both look like a working rule. Routing fails silently, which is the scenario. And the reader fails silently, in the sense that an analyst closing a true positive as benign leaves a record that says the alert was handled.

Only the query announces itself, and only at the moment you write it, which is the last moment anybody looks at it closely. Everything to its right reports success indefinitely while producing nothing, which is how a detection estate becomes simultaneously well maintained and inert. Nobody is neglecting it. The maintenance is real, it is competent, and it is aimed at the one stage of five that was never the problem, which is why more of it does not help.

Every detection contains at least one number that a person picked, and the number is usually not defensible from first principles. Five failed sign-ins or ten. A five-minute window or an hour. Two countries or three. Ninety days of baseline or thirty.

Those numbers determine everything about how the rule behaves and they are almost never documented with a reason. A rule inherited from a template arrives with somebody else's numbers, chosen for somebody else's estate, and the fact that it fires plausibly often in yours is taken as evidence they were right.

The uncomfortable part is that there is frequently no correct answer. A threshold trades false positives against false negatives and the exchange rate depends on how much analyst time exists, which is an operational fact rather than a security one. The same rule with the same threshold is well tuned in one organization and useless in another, and nothing about the rule tells you which you are in.

One class of threshold is worth treating differently: the public ones. A number from a well-known template or a vendor default is a number an adversary can stay under. Smart lockout at ten failures in sixty seconds is documented, so a spray at two attempts per account per day is calibrated against a published figure rather than against your estate.

That does not make the default wrong, and changing it for secrecy is not a strategy. It means a public threshold needs a companion keyed on a different dimension: the same attack seen as distinct accounts per source address rather than failures per account.

What you can do is know which direction each number moves you and record why you chose it. A rule whose threshold has a written reason can be revisited when the estate changes; a rule whose threshold is a number somebody typed in 2023 cannot, because nobody can tell whether changing it would break something.

id04 returns to thresholds as a tuning problem with real numbers attached, and id05 returns to them again from the operational side, where the question is how much analyst time a threshold is spending. What matters at this stage is recognizing that every rule contains at least one number somebody picked, that the number is usually undocumented, and that a rule inherited from a template arrives carrying decisions made for an estate that is not yours.

Routing is the part with the least written about it and the most ways to fail, because it is not one setting. It is the severity, the incident configuration, the queue filter the analysts actually use, the working hours of whoever owns that queue, and whether anything downstream is subscribed to it.

In the scenario a rule was created with severity Informational, which is a default rather than a decision. The analysts work a queue filtered to Medium and above, which is a sensible filter for a team of three taking sixty alerts a day. Both of those are defensible in isolation and together they produce a rule that has been correct three times and read zero times.

Five ways routing fails, each invisible from the rule. Severity below the queue filter, which is the scenario. Incident grouping folding the alert into a larger incident whose title describes something else. Alerts generated without incidents at all, which is a separate toggle. A queue owned by a team whose hours do not cover when the rule fires. And nothing downstream subscribed, so it fires into a system with no next step.

Connect-AzAccount

Get-AzSentinelAlertRule -ResourceGroupName "rg-sec" -WorkspaceName "ne-sentinel" |
  Where-Object { $_.Enabled } |
  Select-Object DisplayName, Severity |
  Group-Object Severity |
  Select-Object Name, Count

# Name           Count
# ----           -----
# High               9
# Medium            31
# Low                4
# Informational      3   <-- below the queue filter, invisible

Three rules in that last row, and the rule from the scenario is one of them. This takes a minute to run and appears on no dashboard, because severity is not something a rule health report treats as a health property. Nor is it visible in the rule list in the portal, which shows name, tactics and last-run status.

The general shape is worth naming because it recurs. Every part of routing is somebody else's reasonable local decision, and the failure emerges from the combination rather than from any one of them being wrong. Nobody set out to hide a privileged role assignment alert. A severity default met a queue filter and the result was invisible to both the person who set the default and the person who set the filter.

That is why id05 treats routing as an operational design question with its own sub rather than as a field on a rule form.

Below is the rule from the scenario as it exists in the workspace, with its definition, its settings and its firing history together. Every field in it is either correct or a default.

Work the definition and the settings against each other rather than reading down the page, because the faults are in the relationship between them rather than in either alone. The reason that audit is harder than it looks is that the query genuinely is fine. There is nothing to find in the eleven lines everybody's eye goes to first, which is the point: the faults are all in fields that a rule review does not treat as part of the rule. Severity, incident grouping, the description that would have carried the threshold reasoning, and the entity mapping that would have let an analyst pivot from the alert to the account.

Two of the faults in that rule are worth pulling out now because they recur throughout the course. The missing entity mapping means an analyst opening the alert sees a row of text rather than a clickable account, so pivoting to that user's other activity is a manual copy and paste at the exact moment speed matters. id04 has a full sub on entity mapping for this reason: it is the difference between an alert that starts an investigation and one that starts a search.

The empty description is the second. It is the only place the threshold reasoning could have lived, and its absence is why nobody in eighteen months could decide whether the rule was tuned correctly. A rule with no stated intent cannot be reviewed, only re-derived.

Section 0.9 returns to this with a coverage argument. For now the useful takeaway is that reviewing detections by reading their queries is roughly as effective as reviewing an aircraft by reading its engine specification.

If four of the five parts fail silently, then writing the query first is starting at the only stage with a safety net, and it encourages treating the rest as configuration to be filled in afterwards.

The order that works better is to establish the data before anything else, because per section 0.2 it is the ceiling on what is possible and it takes ten minutes to check. Then decide who reads the result and what they will do about it, which is a conversation rather than a task and frequently changes what is worth writing at all. Then choose the threshold, with a reason recorded. Then write the query, which by that point is the mechanical part.

That order feels backwards to anybody who enjoys writing queries, and it is deliberate. A rule nobody will act on is not worth writing correctly, and finding that out before the forty minutes rather than after is the whole benefit.

That order also changes what gets built. Starting from the data means you discover early that a whole class of rules is impossible in your tenant, which is useful information rather than a disappointment. Starting from the reader means some ideas die in the conversation: a rule that would fire eight times a day into a three-person queue is not viable however good its logic, and knowing that before writing it saves the forty minutes and the argument later.

It is worth saying that this is not how most teams work and not how most people enjoy working. Writing the query is the satisfying part, and the four other decisions are meetings. The estate in section 0.1 was built by people doing the satisfying part well.

The scenario's failure is common enough that most estates have at least one, and it takes a single query to find.

The next section turns from the mechanics of a detection to what it is looking for, and specifically to the attacks that succeed against a tenant with every prevention control switched on.