0.5 What This Course Builds

Introduction

The four lessons before this one were the course in miniature: an estate, its evidence, an assessment of the rules that already ran in it, and one detection written in every form. The rest of the course does each of those properly, at the scale of a detection program rather than a handful of rules, and ends with a case you work alone.

This lesson lays out what that involves, from the course's own files rather than a description of them: how many modules and lessons, which kind each lesson is, what you practice and how often, the tools, the library of rules you finish with, the resources, the Project and the exam. Every count here was taken from the lessons themselves when this page was built.

The figure is the route. The sections say what is at each stop.

The course follows a detection engineer's work in the order it is learned. Each phase depends on the one before it, and each module's last lesson is where its pieces are assembled. The modules, by phase, with their lessons counted from the module directories:

Foundations
   1  Introduction to Detection Engineering                5 lessons
   2  Adversary Behaviors and Threat Modeling              4 lessons
   3  Security Telemetry and Logging Fundamentals          6 lessons
   4  Security Data Engineering and Normalization          5 lessons
Development and query engineering
   5  Detection Hypothesis and Rule Design                 7 lessons
   6  KQL for Detection Engineering                        5 lessons
   7  Sigma and Portable Detection Rules                   5 lessons
   8  Correlation and Behavioral Analytics                 5 lessons
   9  SIEM and XDR Detection Implementation                5 lessons
Threat-focused detection
  10  Windows Endpoint Detection Engineering               6 lessons
  11  Linux Endpoint Detection Engineering                 5 lessons
  12  Identity and Cloud Detection Engineering             6 lessons
  13  Network Detection Engineering                        4 lessons
  14  Email, Ransomware and Multi-Stage Attack Detection   5 lessons
Validation, automation and operations
  15  Detection Testing and Adversary Emulation            5 lessons
  16  Detection Tuning and Performance Optimization        5 lessons
  17  Detection-as-Code and CI/CD                          5 lessons
  18  Detection Operations, Quality and Coverage Management 5 lessons
Advanced techniques
  19  Advanced and Emerging Techniques                     5 lessons

98 lessons in all. The first phase is what a detection is and what it needs: reading one, the threat behind it, the telemetry that records it, and the data engineering that makes that telemetry readable. The second is building one: a specification with tests, the query in KQL and SPL, a portable Sigma rule, correlation across sources, and deployment in Sentinel and Defender XDR.

The third phase applies all of that to each attack surface in turn, Windows, Linux, identity and cloud, network, and email with ransomware, each module a pack of detections for that surface with their look-alikes.

The fourth runs the result as a program: testing and emulation, tuning, detection as code with a pipeline, and the operations that keep a library honest, inventory, health and measured coverage. The fifth takes the same methods to their edges, anomaly functions, evasion, rules an assistant drafted, and an AI service as an attack surface.

Each module title is also a promise about what you will have at its end: a module's last lesson assembles its pieces into one deliverable, a coverage assessment, a detection pack, a validation report or a tuned rule, which is the module's practical lab. The order matters more than in most courses, because later modules assume the habits of earlier ones without restating them.

A lesson in Module 12 writes its specification, tests its rule on a set and schedules it as Modules 5, 6 and 9 taught, and spends its words on the identity attack instead. Moving through the first two phases quickly is reasonable for an experienced engineer; skipping them usually means meeting their ideas later, mid-lesson, without the explanation.

Every lesson is one of two kinds, and you can tell which from its first section. A detection lesson builds one rule through the whole lifecycle, from the attacker's behavior to monitoring the rule in production. Its sections, from the password spray lesson in Module 12:

12.1 Password Spraying from One Address and from Many
  1. The Threat: A Spray Spread over 23 Addresses
  2. The Hypothesis: What the Rotation Can't Spread
  3. The Logic: Count the Tenant, One Day at a Time
  4. Testing: Both Test Sets, Both Rules
  5. The Logic, Continued: The Password That Worked
  6. Tuning: The Threshold and the Window
  7. Response: What Each Alert Asks the Analyst to Do
  8. Deployment and Monitoring: Two Rules That Cover Each Other

A task lesson does one piece of engineering work on something that already exists, a measurement, a review, a repository, an assessment like lesson 0.3's, and runs Situation, Do and Verify. From Module 15:

15.4 Measuring a Rule
  1. The Situation: Three Rules, Two Versions Each
  2. Do: Fix the Vocabulary
  3. Do: Hits and Misses, Before and After
  4. Do: Read What Precision Counts
  5. Do: Measure the Arrival Half of Latency
  6. Do: Measure the Schedule Half
  7. Do: Measure One Rule Yourself
  8. Verify: One Sheet for Three Rules

The two kinds share their standard. Each section opens and closes on explanation rather than on a code block, every query in it runs in the page, every number in it came from running something, and each lesson ends with a practice task that does the lesson's work again on something new. Modules mix the two according to their subject.

Every lesson in Module 12, identity and cloud, is a detection lesson, one attack and its rule each, and every lesson in Module 15, testing and emulation, before its summary is a task lesson, one piece of the testing work each, because its subject is the work around rules rather than any one rule.

The detection spine also says what each section is for, which helps when you come back to a lesson for one thing.

The telemetry section is where the data a rule needs is established, the testing section is where it is scored on its set, and the deployment section is where its schedule, its entity mapping and its severity are decided, so a reader looking for how one rule is scheduled goes straight to that section.

Detection lessons in Module 7, where Sigma is taught, and from Module 14 onward carry the rule's Sigma form beside its KQL and SPL; Modules 8 to 13 keep to KQL and SPL, so their lessons spend the space on the attack.

Task lessons carry no Sigma form, because their subject isn't a rule. That split decides where to look for something later: a detection's reasoning is in its detection lesson, and the engineering around detections, how they are measured, reviewed, deployed and kept, is in the task lessons.

Reading about detection engineering and doing it are different skills, so the lessons are built around things to operate rather than things to read. Counted across the 98 lessons:

runnable KQL and SPL blocks           872
graded exercises                       74   62 on a test set, 1 tuned, 11 on a result
test run boards                        25
rule and plan reviews (auditor)        23
decisions with consequences            11
knowledge check questions             152

The runnable blocks are the largest number because every query on every page is one: run it as it stands, change it, run it again. The graded exercises are where the course checks your work.

A test-set exercise gives you a rule that is wrong in a specific way and scores your fix on the labeled cases, recall and look-alikes together, so the only way to pass is to catch the attack without catching the administration.

A tuning exercise gives you a noisy rule and the knobs, and rescores it at every move. A result exercise checks a value, a count or an entity, that only a correct query finds.

The exercises are where most of the course's time should go, because they are where a wrong idea fails visibly. A rule that reads well and misses its case, or catches its case and the look-alike with it, shows up as a score rather than as an opinion, and the hints show the path without giving the rule away.

The test run boards are the comparisons lessons make between versions of a rule, each cell scored on the corpus when the page was built. The reviews are rule files and plans with planted faults that you find, classify and fix, the work a reviewer does on a pull request.

The decisions are calls with real trade-offs, what to detect first or whether to accept a gap, where the consequence of each choice is shown after you commit to it.

Every module closes with a knowledge check and a challenge set against the corpus, 19 sets in all.

A challenge set is a list of questions whose answers are values, a count, a date, an account, found by writing the query yourself, which makes it the closest thing in a module to the exam, and the best measure of whether the module's method has stuck before you move to the next.

The knowledge checks test judgment rather than recall: each question is a situation from the module with four defensible answers, and the explanation for every option says what it gets right and what it misses.

Four instruments do the measuring, and each appears where the work first needs it:

Testing Workbench    a rule run on a test set, scored          62 exercises
Tuning Simulator     a threshold or exclusion moved, rescored  1, first in 16.2
Coverage Mapper      mapped coverage against tested coverage   first in 2.3
Quality Checker      a rule file reviewed against the standard first in 5.6

The Testing Workbench is the graded exercise block: the scoring lesson 0.2 showed on a board, run by you on a rule you wrote: write it, run it, and see which malicious cases it caught, which look-alikes it flagged and what else it returned.

The Tuning Simulator is the same scoring with knobs, a threshold or an exclusion to move, and the score redrawn every time, so you see the moment a change removes noise and the moment it removes the attack.

The Coverage Mapper separates two numbers that almost every coverage report conflates: techniques a rule is tagged with, and techniques a rule was shown to catch on labeled data. Lesson 2.3 starts the map and 18.3 finishes it, with the gap between the two numbers as the finding.

The Quality Checker reviews a rule file against the course's rule standard: its metadata, its ATT&CK mapping, the fields it needs, its tests and its documentation. You use it as a reviewer in the lessons and as a gate in Module 17's pipeline, where the same checker runs on every pull request.

The four answer the four questions a detection engineer is asked about any rule: does it work, can it be made quieter without breaking it, what does it cover, and is it written well enough for someone else to maintain. A program that can't answer each of those for each rule is running on trust.

None of the four is a demonstration. Each reads the same labeled corpus and the same rules the lessons build, so a result in a tool is a result about the rule, and the scores in the tools are computed by the same code that scored this module's rules.

The rules the course builds don't stay in the lessons. They accumulate in a repository, ne-detections, which Module 17 builds and Module 18 fills, with a pipeline that validates and tests every change before it can deploy. At the end of Module 18 it holds:

rules/windows      10
rules/identity      4
rules/network       4
rules/azure         2
rules/correlation   2
rules/sentinel      2
rules/aws           1
rules/email         1
rules/m365          1
hunts/              6
sigma/rules/        5
tests/acceptance.yml, tools/validate.py, tools/test_rules.py, tools/build.py

27 scheduled rules across the estate's surfaces, 6 hunts that run on demand rather than on a schedule, and the Sigma pack from Module 7. The scheduled rules carry 21 ATT&CK techniques between them, each with a test set and an acceptance entry, so the library's coverage is a tested number rather than a tagged one.

The tools are the ones lesson 0.4 ran: validate checks each file against the schema, test_rules scores each rule on its set and on its schedule, and build turns the files into the templates a deployment uses.

The repository is the course's answer to the problem lesson 0.3 started with. An inherited rule set is hard to assess because nobody wrote down what each rule was for or how to tell whether it worked.

A repository in which every rule carries its specification, its test set and its bar can be assessed by running one command, and a rule that stops working fails a test rather than going quiet.

Each rule in it was built in a lesson, tested on its own labeled cases, and moved into the library through a pull request, the way Module 17 sets out, so its history says who changed it, when and why, and its tests say what it was shown to do.

You build your own as you go. The practice tasks write rules in the same format, and Module 17's lessons show how to host the repository and its pipeline, on GitHub or anywhere else, with your rules in it rather than Northgate's.

A course is read once; its reference material is used for years. Eight resource modules sit after the teaching modules, numbered from 90:

90  Cheatsheets   one sheet per teaching module, each entry run on the corpus
91  Cookbooks     advisory to rule, rule to production, tuning, retiring a rule
92  Lab Setup     the corpus and the tools, and an optional Sentinel workspace
93  Walkthroughs  worked cases, each from a report to a validated rule
94  Playbooks     a silent rule, a noise spike, a telemetry gap, a schema change
95  Playground    the consoles and tools, and what to practice in each
96  Library       every rule the course builds, in KQL and SPL, and Sigma
97  References    sources by question, link-checked

The cheatsheets and the library are the ones most people return to. A cheatsheet holds one module's queries and patterns, each run on the corpus when it was built, for the moment you know what you need and want the syntax.

The library is every rule the course builds in one place, in KQL and SPL with its Sigma form where its lesson wrote one, and its test set, for the moment you need a rule and want to start from a tested one.

The cookbooks and playbooks are procedures, written to be followed under time pressure rather than read through. A cookbook takes you through the engineering of a rule from an advisory to production, through tuning or retirement.

A playbook is what to do when a detection fails in production: a rule that went silent, a sudden spike in noise, a telemetry gap, a schema change upstream. Those four are the commonest ways a working detection stops working, and the course teaches each in a lesson, so the playbook is the lesson's method distilled to the order you do things in.

Lab Setup covers the corpus and the tools, where each one runs and what it reads, and an optional Sentinel workspace of your own, with what it costs from Microsoft's current pricing. The walkthroughs are worked cases from a threat report to a validated rule.

The references are the sources the lessons rely on, grouped by the question they answer and checked so the links still work. Vendor documentation changes faster than any course, and a reference that is grouped by question tells you which page to recheck when a product changes, rather than leaving you to remember where a fact came from.

The course ends with a Project: a prepared case from another organization, a multi-stage identity and endpoint compromise in data you haven't seen, with the complications real work brings. Some telemetry is missing, some administration looks like attack, some data arrives late, and one person appears under different identifiers in different sources. Nothing in it has been worked for you.

You deliver what a detection engineer would deliver to the people in that case:

five rules       complementary, covering the compromise between them
their tests      each rule on its malicious case and its look-alikes
a tuning record  what each rule fired on first, and what changed it
a repository     the rules, their tests and acceptance entries, as Module 17
guides           one investigation guide per rule, for the analyst it alerts
a coverage page  what the five catch, what they miss, and why

The five rules are complementary by design. No single rule sees the whole compromise, because the case's complications put each stage in a different source and some stages in none, so the coverage page is where you show how the five together close the gaps any one of them leaves, and say plainly which gaps remain.

There is no submission, review or grade, and the platform never sees the work. The Project closes with a checklist built from the criteria every rule in the course meets: it catches its malicious case, it has tests for its look-alikes, and its limitations are stated.

The exam is separate, shorter and graded, and it is what the certificate rests on. It is one task per teaching module, nineteen in all, each answered with a value you find by running a rule or a query against the corpus: how many malicious events a rule missed, a precision figure, the entity behind an alert.

It is marked out of a hundred, with seventy to pass, and passing earns the course certificate, which carries the course's CPE. An exam answered by running queries can't be passed by recognizing the right option, which is the point.

The two test different things. The exam asks whether you can apply each module's method to a question with one right answer. The Project asks whether you can choose which methods a case needs and deliver work someone else could rely on, where no answer key exists.

The course is 38 to 40 hours, and completing it earns 40 CPE. The hours aren't spread evenly. Measured by teaching words, the share of the course each phase holds:

phase                                    modules   share of the teaching
Foundations                              1-4        22 percent
Development and query engineering        5-9        29 percent
Threat-focused detection                 10-14      27 percent
Validation, automation and operations    15-18      18 percent
Advanced techniques                      19          5 percent

The second and third phases hold 56 percent of the teaching between them, so a pace set in the first phase will feel slow in the middle and quicker again at the end. Plan the middle ten modules as the bulk of the time, and the fourth phase as shorter but denser, since its lessons work on the repository and the library more than on new queries.

Everything a lesson teaches is done in the page, against the corpus. Two modules involve a platform outside it: Module 9 deploys rules in Sentinel and Defender XDR, and Module 17 deploys them from a repository through a pipeline. Both show the configuration from evidence files and portal views, and a student with a Sentinel workspace of their own can repeat each step there, set up from Lab Setup.

No lesson depends on a workspace, and no result in the course needs one.

The course also never asks you to run an attack. Module 15 teaches adversary emulation, Atomic Red Team tests and replayed telemetry, from the events those tests produced, already captured in the corpus's test workspace. Where you would run a test of your own, the lesson says what isolation it needs and leaves the decision to you and your own lab.

That leaves the route to you, and the Practice task below is where you write it down. A detection engineer already writing rules daily might move through the first phase in an afternoon and spend most of the time in phases three and four.

Someone new to the work gets most from taking every phase in order. Either way the Project at the end asks for all of it, and the Fit Check on this module's index page is a quick way to see where you stand before you choose.

Practice

The route is this lesson's task: which modules you will work closely, which you will move through, and what you want to be able to do at the end.

That completes Module 0. Module 1 begins where lesson 0.3 left off, reading a detection in depth: what it claims, what it needs and what fires it.