TH0.6 Hunting, IR, and Detection Engineering

Three disciplines, one data lake

The tools overlap. Sentinel is the workspace for detection engineering (analytics rules), hunting (hunting queries), and incident response (incident investigation). KQL is the query language for all three. The Defender XDR portal serves all three. A single analyst may write a detection rule in the morning, investigate an incident in the afternoon, and run a hunt campaign on Friday.

The overlap is why organisations confuse them. If the same person uses the same tool to query the same data, what is the difference? The difference is trigger, method, and output — three operational dimensions that separate the disciplines regardless of shared tooling.

Trigger — what starts the work

Detection engineering is triggered by planning. The detection engineer reads a threat report, identifies a technique, writes a KQL query to detect it, tests the query against historical data, and deploys it as an analytics rule. The trigger is proactive but the result is reactive — the rule sits and waits for the technique to occur. The engineer may not be present when the rule fires. The value is in the automation: the rule runs 24/7 without human intervention.

Incident response is triggered by an alert. Something fired — a detection rule, a user report, an external notification from a third party or law enforcement. The IR analyst investigates: what happened, how far did it spread, what data was affected, how do we contain it. The trigger is reactive. The investigation is reactive. The output is a resolved incident. The analyst works until the specific compromise is contained and documented.

Hunting is triggered by a hypothesis. The hunter suspects that a specific threat technique may have occurred in the environment without triggering any alert. They formulate the hypothesis, query historical data across multiple tables, analyse results, and confirm or refute the hypothesis. The trigger is proactive and the investigation is proactive — no alert fired, no incident exists, no evidence of compromise is known. The hunter goes looking for what the rules missed.

The triggers define when each discipline activates. Detection engineering activates during planning cycles. IR activates when something goes wrong. Hunting activates on a schedule — dedicated time, protected from the alert queue, applied to a prioritised backlog of hypotheses. If your analysts only activate when alerts fire, you have one discipline running: IR.

Method — how the work is done

The method differences are as important as the trigger differences, because they determine what each discipline can find.

Detection engineering works by specification. The engineer writes a precise query that matches a specific pattern — "inbox rule created with ForwardTo parameter from an IP not seen in the last 90 days." The query encodes the engineer's anticipation of what the attacker will do. If the attacker does exactly that, the rule fires. If the attacker does something the specification does not cover, the rule is silent. The method is precise, automated, and bounded by the specification.

Incident response works by evidence trail. The analyst starts with a known event (the alert) and follows the evidence forward and backward in time. What happened before this alert? What happened after? Which accounts were involved? What data was accessed? The analyst follows the trail wherever it leads, pivoting across data sources as the evidence demands. The method is adaptive but reactive — it cannot start without evidence of compromise.

Hunting works by hypothesis testing. The hunter formulates a statement about what may be true in the environment — "an attacker who compromised an account via AiTM phishing has established OAuth persistence in the past 30 days" — and designs a query strategy to test it. The hunter is not following an evidence trail because no evidence has been surfaced yet. They are casting a net shaped by the hypothesis and examining what it catches. The method is exploratory, structured by the hypothesis but not constrained to a single event or evidence chain. A hunt can span 30, 60, or 90 days of historical data across multiple tables simultaneously.

// Incident source distribution — which disciplines are producing findings?
SecurityIncident
| where TimeGenerated > ago(180d)
| extend IncidentSource = case(
    tostring(AdditionalData) has "Scheduled", "Detection Rule (Automated)",
    tostring(AdditionalData) has "Fusion", "Fusion/ML Correlation",
    tostring(AdditionalData) has "NRT", "Near Real-Time Rule",
    "Built-in / Other")
| summarize
    IncidentCount = count(),
    AvgSeverity = avg(case(
        Severity == "High", 3,
        Severity == "Medium", 2,
        Severity == "Low", 1, 0))
    by IncidentSource
| sort by IncidentCount desc

If 100% of incidents come from detection rules, your triad has one active leg. Incidents discovered by hunting appear as manually created incidents — or do not appear in Sentinel at all if the team records hunt findings in a separate tracking system. The gap between what rules find and what exists is the hunting surface.

Output — what each produces

Detection engineering produces rules. Analytics rules in Sentinel, custom detection rules in Defender XDR, alert policies in Defender for Office 365. These are the automated layer — they fire when the pattern matches. A detection engineer's work is measured by technique coverage, rule efficacy, and false positive rates. The deliverable lives in the platform and runs without human intervention. A good detection engineering programme produces 5–10 new or significantly tuned rules per month, maintains a coverage map against ATT&CK, and retires or revises rules that have decayed.

Incident response produces findings. Evidence timelines, scope assessments, containment actions, root cause analysis, lessons learned, and IR reports. These document what happened and how the organisation responded. An IR analyst's work is measured by mean time to detect, mean time to respond, containment time, and recurrence rate. The deliverable is a documented, resolved incident. A good IR programme closes incidents with root cause identified, scope fully assessed, and remediation actions tracked to completion.

Hunting produces three things simultaneously. First: findings — evidence of compromise or confirmation of absence. Confirmation of absence is genuinely valuable. When a hunt for AiTM persistence across 30 days returns no results, the organisation has evidence that the technique either did not occur or was not successful. That evidence reduces uncertainty, which has operational value for risk management and for prioritising where to invest next. Second: detection rules — every successful hunt should produce at least one new rule that automates the detection going forward. The hunt query, tested against real production data and tuned for the environment's noise level, is a better detection rule than one written speculatively from a threat report. Third: environmental understanding — baselines, patterns, and contextual knowledge that improves both IR analysis and detection rule tuning. A hunter who spends a week examining OAuth consent patterns across the tenant learns what normal looks like in that specific environment. That knowledge makes detection rules more precise and IR investigations faster.

The detection rule output is what makes hunting self-reinforcing. Each hunt that produces a detection rule moves a technique from the known-unknown layer to the known-known layer. The detection gap shrinks. The next hunt can focus on a different technique because the previous one is now automated. This is the hunt-to-detection pipeline — the mechanism that makes hunting a force multiplier rather than a standalone activity.

The six handoffs

The three disciplines form a cycle with six handoffs. Understanding these handoffs is what separates an integrated security operation from three teams working in parallel without connection.

In most organisations, handoff 1 works reliably because it is the default operating mode: rule fires, analyst investigates. Handoffs 2 through 6 are where the system breaks down. If your IR team closes incidents without feeding gaps back to detection engineering (handoff 2), the same blind spots produce the same surprises quarter after quarter. If your detection engineers identify coverage gaps but never pass them to hunters (handoff 3), the gaps are documented but never investigated. If your hunters find threats but do not produce detection rules (handoff 4), the same technique must be hunted again next quarter instead of being automated.

The practical test for any organisation is straightforward: pick ten closed incidents from the past six months and trace the handoffs. For each incident, ask three questions. Did the investigation identify a technique the attacker used that no detection rule covered? If yes, did that technique enter the detection engineering backlog? And did any hunting campaign run as a result of the investigation's scope questions? If all ten incidents ended at handoff 1 — rule fired, analyst investigated, incident closed — the triad has one active leg. The other two disciplines exist in name but not in practice.

Where hunting sits in the Northgate operation

At Northgate, Rachel Okafor is building the operation to cover all three legs. The SOC analysts handle IR — they investigate alerts, contain compromises, and close incidents. Tom Ashworth is being developed as the detection engineering lead — he writes analytics rules, manages the detection lifecycle, and tracks ATT&CK coverage. The hunting capability is what Rachel is building now. When it is operational, the three disciplines will feed each other: Tom's coverage gap analysis generates hunt hypotheses. Hunt findings produce new detection rules for Tom's pipeline. IR investigations generate scope questions that become hunt campaigns. Hunt discoveries escalate to the SOC for incident response.

The operation Rachel is building is the same operation you build in this course. The Hunt Cycle that structures every hunt campaign — hypothesise, scope, collect, analyse, conclude, convert — is designed around these handoffs. The "convert" step is handoff 4: the hunting query becomes a detection rule. The "conclude" step includes handoff 6: if the hunt finds evidence of compromise, it escalates to IR. Every hunt campaign in Modules 4 through 14 produces both a finding and at least one detection rule, because that is how hunting earns its place in the triad.