What Identity Detection and Operations Teaches

Introduction

This section is the argument for the course. You will finish able to say what separates a control that is switched on from a control that is working, why that gap is a detection problem rather than a configuration one, and why detection and operations are taught here as one subject instead of two.

Northgate would tell you it has identity detection, and it would be true. 214 rules is a deliberate investment and the estate passes any audit that counts controls.

It also lost nineteen days.

The distance between those two facts is the subject of this course, and it is not a gap in the configuration. Nothing at Northgate was misconfigured. The rules were correct, they ran on schedule, and one produced exactly the alert it was written to produce at the moment it was written to produce it. The failure sits somewhere the configuration cannot describe.

You will meet three versions of that distance and they fail differently. A rule that fires into a queue nobody can work has done its job and produced nothing, because a detection is finished when somebody acts, not when it fires. A rule that queries data the tenant does not hold is a rule-shaped object, and it passes every review that reads rules rather than data. And a rule right often enough to be trusted and wrong often enough to be ignored trains the analyst reading it, which is the slower failure and the more complete one.

None of those three is visible from the rule. That is why the course is not organized around writing rules, though you will write a great many of them. It is organized around the question of whether the thing you built is doing anything, which is a different question and a harder one.

That is not an argument against measurement, and this course spends a module on it. It is an argument about which measurement, and the answer is usually the one that is harder to collect.

Consider what Northgate could report that morning, all true: 214 rules enabled, 47 identity, 94 per cent triaged inside target.

None is a statement about whether an attack would be caught. They describe effort, coverage and throughput, the three things easiest to count and least connected to outcome. The number that mattered was how many of the 47 identity rules produced a true positive in ninety days, and nobody kept it, because it is the only figure there that can go down.

The rule that fired on day two was a good rule. It looked for a sign-in from an address with no prior history for that account, on a session that had not re-authenticated, and it was written by somebody who understood token replay. On the day it mattered it produced an alert inside twenty-four hours of the compromise.

It arrived in a queue running sixty alerts a day, most of them impossible travel from the VPN egress. The analyst working it had closed forty of those in a week and been correct every time.

The shape of the noise is what defeated the signal. Northgate routes every remote session through one VPN egress address, so a user at home, a colleague in a hotel abroad and anybody on the corporate network all appear at the same place. The distance between two consecutive sign-ins is therefore frequently impossible, for a reason that has nothing to do with an attacker.

That is a tuning problem with a known fix: exclude the egress address from the location logic. It was raised twice and recorded as low priority both times, because the alerts were being closed correctly and the queue was clearing inside its target, so by every operational measure the process was healthy.

The fix is not more diligence and it is not a stern word about alert fatigue. It is that the sixty daily impossible-travel alerts were a tuning problem somebody had decided not to solve, and every one of them made the next real finding fractionally less likely to be read. Detection tuning, which sounds like housekeeping, is the thing that decides whether your good rules are audible.

You will spend a full sub of id04 on exactly this, and the argument there is uncomfortable: a rule with a high false-positive rate does not merely waste time, it actively degrades every other rule sharing its queue. The cost is not local to the noisy rule. That is why tuning is treated in this course as a detection engineering activity rather than an operational chore.

The second rule is the more interesting failure, because it would have caught the compromise a day earlier and there is no version of the story where it fires.

It queried AADNonInteractiveUserSignInLogs, the table that records token refreshes and silent authentication rather than interactive sign-in. That is the right table: an attacker holding a stolen session token never signs in interactively, so they never appear in SigninLogs, and the only record of their access is the refresh activity. Whoever wrote that rule understood the attack better than whoever wrote the first, and in ninety days it has never returned a single row, which nobody noticed because a rule returning nothing is not an error and does not appear in any health report.

Northgate does not ingest that table, and has not for eight months. It is the highest-volume identity source in a typical tenant, often ten to fifteen times the interactive volume, and somebody reviewing workspace cost switched it off. The decision was made by a person with a budget, on a table whose name gives no hint of what it holds, and nobody was asked what would stop working.

The decision was not unreasonable. An interactive sign-in happens a handful of times a day. A non-interactive one happens every time a client refreshes a token, which for a user with Outlook, Teams and OneDrive open is every few minutes across three applications. Across 810 people that compounds into a table an order of magnitude larger than the one everybody knows about.

The table below is what an ingestion review sees, and beside it what a rule review sees. Neither team looks at both, which is the whole of how the second rule died. Read the two halves against each other rather than in order.

Log Analytics | identity tables, 30 days, Northgate

TABLE                                 ROWS        INGESTED
SigninLogs                         128,400        yes
AuditLogs                           41,900        yes
AADServicePrincipalSignInLogs      486,700        yes
AADNonInteractiveUserSignInLogs          0        NO

Analytics rules referencing each table
SigninLogs                              31
AuditLogs                               12
AADServicePrincipalSignInLogs            0
AADNonInteractiveUserSignInLogs          4  <-- enabled, cannot return

The bottom two lines are the finding and neither of them appears in any rule review, because a rule review reads rules. Four enabled rules query a table holding nothing, and no rule at all queries the service principal table, which is ingesting half a million rows a month for no consumer.

Connect-MgGraph -Scopes "Directory.Read.All"

Get-MgBetaEntraDiagnosticSetting |
  Select-Object -ExpandProperty Logs |
  Where-Object { $_.Enabled -eq $true } |
  Select-Object Category

# Category
# --------
# SignInLogs
# AuditLogs
# ServicePrincipalSignInLogs
#
# NonInteractiveUserSignInLogs is absent, which is the answer.

That question is configuration rather than events, so it is a PowerShell question and not a KQL one. A query against the table tells you what is in it; only the diagnostic setting tells you whether anything is being written to it at all, and a table that is empty because nothing is routed there looks exactly like a table that is empty because nothing happened.

Two conversations produced this. A workspace review found one large item with no obvious consumer. A rule review found 47 rules enabled and correctly written and reported the estate healthy. Both accurate. The only artifact showing the problem belongs to neither team.

So the person reviewing the bill saw a large line item on a table with an opaque name and no obvious consumer, and switched it off. Every part of that was sound except the part nobody supplied: four analytics rules depended on it.

A rule is a claim about data that exists. When it is not, the rule still parses, still saves, still appears in every count of enabled rules, and still shows green in every dashboard that reports rule health, because rule health means the rule executed without error and a query returning nothing is not an error.

That is the second failure mode and it is invisible to review. You cannot find it by reading rules, however carefully. You find it by checking the tables your rules depend on against the tables your workspace receives, a ten-minute exercise almost nobody performs and which nothing in the product prompts you to do.

There is a reasonable version of this material split in two, a detection course and an operations course. It was nearly built that way and the scenario is the reason it was not.

Look at where each failure lives. The first belongs to detection engineering, because tuning a noisy rule is rule work. The second belongs to operations, because ingestion is a workspace decision made in a diagnostic settings blade by somebody managing a data budget. Neither team could have found the second failure alone. The detection engineer does not see the ingestion configuration and has no reason to look at it; the operations engineer sees a table with a cost and no visible consumer.

That is the argument for one course rather than two, and it is worth stating plainly because the alternative is defensible and common. Most organizations do run these as separate functions with separate reporting lines, and the scenario is what that costs. You are being taught both halves so that you can see the decision the other side is making.

The same seam runs through the rest of the material. A workload identity has no interactive session, so whether you can detect its compromise depends entirely on whether the service principal sign-in table is being ingested, which is an operations decision. An access review that nobody acts on produces a governance record and no change, which is the queue problem in a different costume. A backup of your Conditional Access policies is worthless until somebody has tried restoring one, which is the difference between a documented control and a tested one, and it is the same difference as between an enabled rule and a rule that has fired.

Every module in this course has that shape somewhere in it: a thing that exists, and a separate question about whether it does anything. You will find it easier to spot after id04 and harder to un-see afterwards.

It is also why detection comes early rather than in subject order. It is the lens the rest of the material is read through, and a reader who meets governance or recovery first has no way to judge what either is for.

Everything above is the argument. This is the evidence it was drawn from: Northgate's identity rule inventory as it stood on the day of the compromise, with what each rule fired and what happened next.

Work through it before reading on. The question is not which rule was missing, because none was missing. It is which of the rules present was the finding, and what stopped it from being read as one.

Two things to watch for. The rules are not equally useful and the inventory cannot tell you which is which. And several overlap, so more than one fires on the same event, which sounds like redundancy and is closer to noise.

There is a method for reading an inventory like this. Table column first, because it says whether the rule can produce anything. Fire count second, because a rule that never fires and one that fires daily are suspicious for opposite reasons. The name last or not at all: it describes what somebody once intended, and it is the part a reviewer reads first.

Two rules in that list are the ones from the scenario. The first fired and was closed as benign. The second could not fire at all, and the inventory gives you exactly one clue about which it is, in a column most reviews never read: the table it queries. Everything else on the row, the name, the severity, the tactic, the enabled flag, is identical between a rule that works and a rule that cannot. That is worth sitting with, because it is why an inventory review is not an assurance activity however carefully it is done.

The inventory is the shape of a real identity detection estate and it is worth noticing what it is not. It is not thin. It is not neglected. Somebody has enabled the Microsoft-supplied identity templates, written several custom rules, and kept them running. On any metric that counts rules, Northgate is doing well, which is the point the scenario exists to make and the reason the first section of this sub is about the difference between a count and an outcome.

By the end you will be able to take an identity estate you have not seen and answer the question Northgate's review could not: is anything here catching anything. That work spans nine modules.

What this course does not do is teach you to design the controls themselves. Conditional Access architecture, authentication method strategy and privileged access design are a different discipline with their own course. Where a control appears here it is explained to the depth the detection needs. You do not need to have taken that course and nothing here assumes it.

Working one estate throughout matters here. Detection is accumulated context, and changing company every module would keep resetting what normal looks like. By id04 you will know Northgate routes remote sessions through one egress address and runs eleven service principals nobody rotates, and those facts turn a generic rule into a rule for this tenant.

The habit this course builds is narrow and it is the one Northgate's review lacked: when you are told a control is in place, ask what it caught, and if the answer is nothing, ask whether it could have.

That habit is uncomfortable to hold. Asking what a control caught invites the answer "nothing, and that is good news", which is sometimes true and is also what a dead rule produces. Telling them apart takes evidence, and gathering it is most of this course.

One question, asked of an estate you have access to, and it will take longer to answer than you expect.

The next section takes the four tables this one kept referring to and says what each of them actually holds. SigninLogs is the one everybody queries and it is the smallest; the non-interactive table that killed the second rule is the largest by an order of magnitude; the service principal table is where workload compromise lives and Northgate ingests it while querying it with nothing at all. AuditLogs is a different kind of source again, recording change rather than authentication, which is why five of this module's detections key on it. Knowing which one answers which question is the difference between a rule that can produce a finding and a rule that is correct and inert.