0.4 The Maturity Spectrum

Introduction

People talk about SOC maturity as if it were one number, usually a level from one to five: a young SOC is immature, a well-funded one mature, and the difference is budget and tooling. That picture is easy to hold and hard to use, because it says nothing about what to change on Monday. This sub uses a narrower definition that the rest of the course builds on: a SOC's maturity is what it does with its own records. Every SOC writes records, because its tools write them whether anyone asks or not. What separates levels of maturity is whether those records are complete, whether anyone reads them, and whether reading them changes anything. Each of those can be checked with a query, and this sub checks Northgate's.

The diagram rises from left to right, and every box refers to the same records. A reactive SOC and an improving one might run exactly the same products and hold exactly the same tables; the difference is in the habits around the tables. That's what makes the definition useful: it points at habits, which a team can change in a month, rather than at products, which take a budget cycle. It also makes maturity checkable by anyone with read access, including a new analyst in their first week, which is the position this course assumes you might be in.

Published maturity models exist, and some are detailed, scoring a SOC on dozens of aspects of people, process and technology. They're useful for a full assessment, and they share a limitation for day-to-day work: they take time to run, and they tend to measure whether a capability exists rather than whether it's used. This course uses four plain levels instead, each defined by what happens to the records.

The definitions are deliberately short, because each has to be checkable, and deliberately about records, because records are the one thing every SOC has. "Every incident has an owner" can be counted; "the SOC has good ownership" can't. The levels build on each other, and in a fixed order. A SOC can't measure what it hasn't defined: a time to close means nothing if closures aren't classified, and a backlog means nothing if incidents have no owner. And a SOC can't improve what it doesn't measure: a change made without a measure before and after it is a guess, however sensible it sounds. The order is the reason each level matters; skipping one produces numbers nobody trusts, or changes nobody can evaluate.

Applied to Northgate, the difference between a label and a placement is plain.

Northgate's SOC: immature
 
One label for all its work

Queue: reactive
Verdicts: defined
Data: reactive
Workers: defined, just
Changes: reactive
 
One line per question

The right-hand pane is what this sub builds, one question at a time. The levels also describe work, not organizations. Most SOCs sit at different levels for different kinds of work: defined for the incident types they see every day, reactive for the rare ones; measured for time, reactive for data health. A single label for the whole SOC hides exactly the differences that matter, which is why this sub places Northgate question by question. It also explains why two people can describe the same SOC as mature and immature and both be right: they're describing different work.

Placing a SOC needs questions whose answers are in its records, so that the answer is a fact rather than a view. Five questions cover the ground this course teaches, and each points at the table that answers it.

Five is enough because each question maps to one of the four functions from Section 0.2 or to the improvement loop that joins them. The first three are about the SOC's basic records: the queue, the verdicts, and the data that feeds both. The fourth is newer and increasingly important: much of a modern SOC's work is done by automation rules and agents, and a SOC that checks its people but not its software has left most of its closures unexamined. The fifth is the one that separates the top level from the rest, because it asks about the SOC's own behavior over time rather than about any single month. A SOC can be measured on all four of the others and still not improve, if its measures are read, filed and never acted on.

Written out, the reactive and measured answers to each question look like this.

Each question has a reactive answer and a measured one, and none of the measured answers needs a tool the reactive SOC lacks; the table is the same, and so is the query. The difference is always the same: whether someone reads the record on a schedule and would notice if it changed. The next four sections ask the first four questions of Northgate's month; the fifth needs more than a month of history, and Section 07 answers it from what the month shows.

The first two questions come from the same table, and one query answers both, which is part of why they're the easiest to start with in any SOC, including your own. It reads each incident's latest version and sets aside the incidents Microsoft XDR resolved on its own, since those were never the SOC's work.

SecurityIncident
| summarize arg_max(TimeGenerated, *) by IncidentNumber
| where not(Status == "Closed" and ModifiedBy == "Microsoft XDR")
| summarize Incidents = count(), FirstTouch = countif(isnotempty(FirstModifiedTime)),
    Closed = countif(Status == "Closed"),
    Classified = countif(Status == "Closed" and Classification != ""),
    Open = countif(Status != "Closed"), Untouched = countif(Status == "New")

Two hundred and sixty incidents the SOC dealt with in the month, of which 201 were touched at least once, 157 closed, 129 of those with a classification, and 103 still open, 102 of them never touched. Each number answers part of a question, and the fractions matter more than the totals, because a total grows with the business and a fraction describes the habit. The verdict question looks respectable: 82 percent of closures say what they were. The queue question doesn't: all but one open incident is untouched, and the oldest has waited the whole month.

Placed side by side, the same numbers reported and the same numbers read tell very different stories, and only one of them says what to do.

Queue: 103 open
Verdicts: 157 closed
 
A month, as reported

Queue: 102 of 103 open never
  touched
Verdicts: 129 of 157 closures
  say what they were
 
A month, as read

The left-hand pane is the kind of summary a reactive SOC produces every month: counts of what happened, all accurate, none placing the SOC. The right-hand pane asks what fraction of each record is complete or read, and those fractions are what place it. On the queue, Northgate is reactive: nobody knows what is waiting without going to look, and the evidence is that nobody went. The untouched incidents aren't all routine, either, and grouping the High ones by the product that raised them shows what they are.

let Untouched = SecurityIncident
    | summarize arg_max(TimeGenerated, Status, Severity, AlertIds) by IncidentNumber
    | where Status == "New" and Severity == "High"
    | mv-expand Id = parse_json(AlertIds)
    | project IncidentNumber, Id = tostring(Id);
SecurityAlert
| join kind=inner Untouched on $left.SystemAlertId == $right.Id
| summarize Incidents = dcount(IncidentNumber) by ProviderName
| sort by Incidents desc

Nineteen High incidents, most of them from Defender for Identity and Entra ID Protection, three from Defender for Endpoint: directory attacks, a stolen session, the ransomware. None was hidden; each sat in the same queue as everything else, sorted by arrival, and waited because the queue was read only when someone felt it was long. A SOC that read its queue daily would have seen each within a day of its arriving. On verdicts it's defined, with a gap: most closures are classified, and the 28 that aren't have never been counted, which is the step to measured.

The fourth question asks whether the SOC's software is checked, the question most maturity discussions still leave out, and the first thing to know is how much of the work the software does. Every incident's first change after creation says who picked it up first, and counting those shows the split.

SecurityIncident
| where ModifiedBy != "Microsoft XDR" and TimeGenerated > todatetime(CreatedTime)
| summarize arg_min(TimeGenerated, ModifiedBy) by IncidentNumber
| summarize Incidents = count() by FirstChange = case(
    ModifiedBy endswith "@ne.com", "a person",
    ModifiedBy startswith "Automation rule", "an automation rule",
    ModifiedBy == "Phishing Triage Agent", "the agent",
    ModifiedBy startswith "Alert tuning", "a tuning rule", ModifiedBy)

The query takes each incident's changes after its creation, sets aside the platform's own resolutions, and keeps the earliest, which says who picked the incident up. Of the 201 incidents anyone touched, a person made the first change on 47. Automation rules made it on 82, assigning and tagging identity incidents seconds after they arrived; the Phishing Triage Agent on 64; and a tuning rule on 8. The person's share is smaller than most readers expect, and it's the share every other measure in this course tends to assume is the whole. Nearly four in five first touches were made by software, and each kind of software made decisions of its own.

The right-hand column is where the fourth question gets its answer.

That split is normal for a modern SOC, and it's what makes the fourth question urgent. Module 13 built the check that found the agent's errors, a month after they happened; nothing yet checks the rules. On the workers question Northgate is defined, just: the software is known, and its output was checked once, after the fact. Measured would mean a sample of each worker's decisions read every week, the tuning rule's closures included, and the automation rules' assignments compared with what their owners then did.

The third question is the one most SOCs answer worst, because data failures are silent. A connector that stops bringing in logs, or a rule that fails its run, raises no alert. Sentinel records each run's outcome in SentinelHealth, and a single count says how much failed in the month.

SentinelHealth
| summarize Runs = count(), Failures = countif(Status != "Success"),
    FailingResources = dcountif(SentinelResourceName, Status != "Success")

Five hundred and five runs of connectors, rules, automation and playbooks, 37 of them failed, across five different resources. Thirty-seven failures in a month isn't unusual; data sources and rules fail in every SOC. The level is set by what happens next. Section 0.3 traced the worst of it: the firewall's connector failing one run in three for more than a week. Nobody fixed it, and nobody knew, because nobody read the table.

This is the clearest reactive answer in the month. The record exists and is complete: every failure is there, with its time and resource. What's missing is only the habit of reading it, and the gap between reactive and measured here is a scheduled query and a named owner. That's typical: the cheapest moves up the spectrum are often the ones that close a silent gap, because the records are already being written. A daily query of failures, sent to whoever owns each failing resource, would have turned the firewall's week-long gap into a morning's, and it would have cost an engineer ten minutes to write.

Three readings of maturity get in the way of seeing that, and each is common.

The first row matters because questionnaires ask people how things work, and people describe the process they believe in rather than the one the records show; Northgate's staff would probably say the queue is triaged daily. The third row is the one this sub has been demonstrating. Northgate isn't a reactive SOC or a defined one; it's defined for verdicts, reactive for data, and somewhere in between for its workers.

With four questions answered from the records, Northgate's place on the spectrum can be written down plainly, one line per question, each with the evidence that put it there. The fifth needs no query of its own: it asks whether any of the findings above led to a checked change, and the month's records show none did. The fifth, whether findings lead to checked changes, is answered from the month as a whole.

Every row carries its evidence, so anyone who disagrees with a placement can rerun the query and argue with the number rather than with the author, which keeps the discussion about the SOC rather than about the people in it. The Changes row is reactive for a specific reason. The month contains findings, a failing connector, an agent's misses, a tail of untouched High incidents, but none of them led to a change that was then checked within the month. Module 13's closure check and Module 11's measures pack are exactly the kind of changes that would move that row, and the course builds both; in the sample month they don't exist yet.

Written this way, the placement is also honest about what's working, and that half deserves its own record.

Each of those is a habit already in place, and each is a foundation for the next level. The classification habit, in particular, is what makes the verdicts question one step from measured rather than two. A SOC that sees only its failures tends to overcorrect, and one that sees only its successes tends to stall; the placement shows both. The record is more useful than a single label because it says where to start. Three rows are reactive, and each has a different cost. The data row costs detection silently; the changes row costs the SOC its future months; and the queue row costs the most right now, because nineteen High incidents are sitting in it unopened.

The instinct after placing a SOC low on the spectrum is to plan big. The proposal below is the common version; make the call before reading where it resolves.

The judgment call recurs in every SOC that has just discovered how much it doesn't know, and the answer is the same each time: start where the records point, not where the model starts. The resolution isn't against assessments; a full model is worth running once a SOC has the habits to act on it. It's against using one to delay the first move, when the records have already named it. A SOC that has moved one question up a level also gets more from a later assessment, because it can show the assessor measures rather than intentions. Moving one question up one level is a small, concrete piece of work, and for the queue question it looks like this.

Four rungs, and none of them needs a new product or a new hire. Together they take about a month, most of it waiting for the second month's reading, and the query from Section 03 is already most of the second rung. The first two make the queue measured: a person reads it on a schedule, from a saved query. The third gives the reading a standard, which is what turns a number into a decision. The fourth is what makes it improving: a month later, the same query shows whether the untouched High count fell, and if it didn't, the owner and the target are the first things to revisit.

Three readings of that ladder undo it, and each is tempting after a first good month.

The rungs are deliberately small, and deliberately in that order. Each can be done by one person in a week, and none needs anyone's permission beyond the SOC lead's. The same four rungs work for every question in this sub, with a different owner and query each time. That's why the course is organized the way it is: each module builds the records and queries for one kind of work, and each ends with a pack that someone owns and reads. By the end of the course, every row in Northgate's placement has the tools it needs to move. None of them needs Northgate to be larger; each needs someone to own a record and read it.

Mapped out, the course's modules move the five questions like this, one or more modules per question.

The three marked rows are the ones Northgate places lowest, and each has more than one module behind it, because a reactive answer usually has more than one cause: a missing record, an unread one, or a reading that never leads to a change. The course doesn't assume a level. A reader in a reactive SOC will find the records and queries to become defined and measured; a reader in a measured one will find the packs and checks that make improvement routine. Most modules move more than one question: Module 11 measures the queue and the verdicts, Module 13 checks the workers, and the hardening and threat intelligence modules move the changes question by giving findings somewhere to go.

Whatever the starting level, the test for whether a question is measured is the same, and it takes a minute to apply.

owner      a name, not a team
schedule   daily, weekly or monthly, written down
query      saved, so the reading is the same each time

The gate is short enough to apply to all five questions in an afternoon, and a SOC that does so has its placement without anyone scoring it. What the course asks of every reader is the habit this sub has used throughout: before deciding how mature a SOC is, or what it should change, read what its records say. Northgate's records placed it in minutes, with a handful of queries, and pointed clearly and specifically at the one change worth making first. Your own records will do the same, and the placement they give is worth more than any score, because each line of it comes with the query that produced it and the step that would move it. The next sub follows a single alert through the pipeline those records describe, from the moment it's raised to the moment its incident is closed.

Practice

Place your own SOC, or Northgate, question by question.

The placement record is the first page of any improvement plan; keep it and redo it next month.

The next sub, Section 0.5, follows one alert through the detection-and-response pipeline, stage by stage.