In this section
Check My Knowledge
Check My Knowledge
Scenario 1: A detection engineer uses Claude to generate a KQL rule from a threat advisory. The rule references a table called DeviceExecutionEvents and the query is syntactically valid. The engineer tests the query in their Sentinel workspace and it returns no results. What is the most likely cause?
The advisory described a technique that does not occur in the engineer's environment, so there are no matching events to return.Incorrect
If the table existed but had no matching events, the query would return an empty result set without error. A syntactically valid query against a non-existent table behaves differently — it fails silently or returns an error depending on the SIEM. The absence of results is the symptom, not the diagnosis. Section 0.2 covers this failure mode under hallucinated references.
Claude hallucinated the table name. DeviceExecutionEvents does not exist in Microsoft Sentinel — the correct table is DeviceProcessEvents. This is the hallucinated references failure mode from section 0.2.Correct
Hallucinated references are the most common AI failure mode in KQL generation. The table name follows Microsoft's naming convention and looks plausible, which is why visual inspection alone does not catch it. The five-check validation requires verifying every table name, field name, and function against the actual workspace schema — Check 1 in the validation discipline from section 0.2.
The time range in the query is too narrow and needs to be expanded to capture the relevant events.Incorrect
A narrow time range would return an empty result set from a valid table, not fail entirely. The question states the query is syntactically valid but returns no results — the distinguishing factor is that the table itself does not exist, which is the hallucinated references failure mode.
The threat advisory described a Linux-specific technique, and DeviceExecutionEvents is a Linux-only table not enabled in the engineer's workspace.Incorrect
DeviceExecutionEvents is not a real table in any Microsoft security product. This option assumes the table exists but is disabled, which would be a configuration issue rather than an AI failure mode. The correct answer identifies that AI invented a plausible-sounding table name — the hallucinated references failure mode from section 0.2.
Scenario 2: Claude generates an incident report draft that includes the statement: "The attacker's behaviour is consistent with a financially motivated threat group operating from Eastern Europe, likely affiliated with the FIN7 cluster." Your investigation identified the attack technique (credential harvesting via AiTM proxy) but not the attacker's identity, location, or group affiliation. How should you handle this claim?
Include it with a caveat — "AI analysis suggests possible FIN7 affiliation, pending further investigation" — since the AI may have identified a pattern the analyst missed.Incorrect
Adding a caveat does not fix the problem. The claim is fabricated — Claude synthesised it from training data patterns, not from your investigation evidence. Including a fabricated attribution with a caveat still presents unverified information in an official report. Section 0.2 covers this under the confident-but-wrong failure mode. Check 3 in the validation discipline requires every factual claim to trace to specific evidence from your investigation.
Replace it with a softer version — "The technique is commonly associated with financially motivated groups" — which is accurate without making a specific attribution claim.Incorrect
While this is factually more defensible, it still introduces an inference that your investigation did not produce. The technique may be common among financially motivated groups, but your investigation identified the technique, not the motivation. Including even a generalised attribution claim without supporting evidence from your investigation adds unverified analysis to the report.
Delete it entirely. The claim is fabricated — Claude has no access to threat intelligence databases or your investigation evidence. Every factual claim in the report must trace to evidence from your investigation. Claims that cannot be verified against your evidence get deleted, not paraphrased.Correct
This is the correct application of Check 3 in the validation discipline from section 0.2. AI-generated attribution without supporting evidence is fabrication presented as analysis. The anti-pattern in section 0.2 describes this exact scenario. AI can help you write the report — structure, formatting, language. Every factual claim must come from your evidence.
Move it to a separate "AI-Assisted Analysis" section of the report so the reader knows it was generated by AI rather than the investigating analyst.Incorrect
Labelling a fabricated claim as AI-generated does not make it appropriate for an incident report. The claim is still unverified. Reports should contain verified findings regardless of the section. Separating AI-generated content creates an expectation that unverified analysis is acceptable if properly labelled — it is not.
Scenario 3: You configure your Claude Project with a system prompt that specifies "Primary SIEM: Microsoft Sentinel" and "Query language: KQL". You start a new conversation inside the Project and ask Claude to generate investigation queries for an account compromise. Claude responds with Splunk SPL queries. What went wrong?
The system prompt is not specific enough — you need to add "Never generate SPL" as an explicit constraint to prevent Claude from defaulting to Splunk.Incorrect
While adding negative constraints can help, the most likely cause is simpler — the conversation was not started inside the Project. The system prompt only applies to conversations created within the Project workspace. Before adding constraints, verify the conversation is running in the correct Project.
You started the conversation outside the Project. The system prompt only applies to conversations created within the Project workspace. Verify you are in the correct Project before troubleshooting the prompt itself.Correct
This is the most common workspace configuration error described in section 0.3. Claude Projects apply their instructions only to conversations started within them. A conversation started from the main Claude interface does not inherit Project instructions. The verification test in section 0.3 is designed to catch exactly this issue — if Claude asks what SIEM you use, the system prompt is not loaded.
Claude's training data contains more Splunk examples than KQL examples, so it defaults to SPL regardless of the system prompt.Incorrect
While Claude's training data distribution could influence default behaviour, a correctly loaded system prompt overrides this tendency. The scenario describes a case where the system prompt is not being applied at all, which points to the conversation being outside the Project rather than a training data bias.
The system prompt needs to be in the Knowledge Base rather than the Project Instructions field for it to persist across conversations.Incorrect
Project Instructions (the system prompt) are the correct location for behavioural directives like platform and language preferences. The Knowledge Base is for reference documents Claude can consult during conversations. Both persist across all conversations within the Project.
Scenario 4: An analyst completes Module 3 on detection engineering and has 15 new prompt templates for advisory-to-hypothesis translation, KQL generation, and rule specification. They plan to study all remaining modules before deploying any artifacts to their production environment. Evaluate this approach.
This is a reasonable approach — completing the full course first ensures the analyst understands the complete methodology before deploying anything to production.Incorrect
Section 0.3 explicitly identifies this as the anti-pattern: studying without deploying. Completing all modules before deploying means 100+ artifacts sitting unused while the analyst's memory of how to apply them fades and the operational environment evolves. The course is designed for deploy-as-you-learn — use each module's artifacts within 7 days.
The analyst should deploy the Module 3 artifacts but only after peer review confirms they are production-ready.Incorrect
Peer review is good practice for detection rules, but delaying deployment until peer review is complete misses the point. The artifacts improve through application — the analyst discovers what works in their environment by using the templates on real advisories. Peer review should happen alongside deployment, not as a gate that delays it.
The analyst should deploy Module 3 artifacts immediately without validation since the course templates have been pre-tested.Incorrect
Course templates are starting points, not finished products. Every template needs adaptation to the analyst's specific environment — workspace schema, naming conventions, false positive profiles. Deploying without validation violates the fundamental course principle from section 0.1: AI handles the mechanical 80%, the practitioner retains the judgment 20%.
This is the anti-pattern from section 0.3: studying without deploying. The analyst should deploy Module 3 artifacts within 7 days — validate them against their environment, apply them to the next advisory that arrives, and refine based on real-world results. Artifacts improve through application, not through additional study.Correct
Section 0.3 identifies deploy-as-you-learn as the core study discipline. The discomfort of using a new technique on live work is the learning signal. The comfort of studying another module instead is the avoidance pattern. Artifacts degrade through non-use because the analyst's memory fades, the environment evolves, and operational confidence never develops.
Scenario 5: You paste sanitised sign-in logs into Claude for investigation analysis. Claude's summary identifies 4 suspicious sign-ins. You verify the first 3 against the raw logs — all 3 are accurate. Should you trust the 4th finding without checking?
No. Verify all 4. AI accuracy on individual claims varies within a single response. Three correct findings do not guarantee the fourth. The validation discipline requires every factual claim to be checked against source evidence — not a sample of claims.Correct
The anti-pattern in section 0.2 describes this exact scenario — trusting partial correctness. Claude may correctly identify 3 suspicious sign-ins and fabricate details about the 4th (inventing a TI attribution, hallucinating an IP geolocation, or misattributing a benign event as suspicious). Every claim is verified independently regardless of how many other claims in the same response were accurate.
Yes — 3 out of 3 verified correctly establishes a high confidence level. Checking every claim in every response is impractical at operational speed.Incorrect
The validation discipline from section 0.2 takes 2 minutes per artifact. Skipping it saves 2 minutes but risks introducing an error that takes hours to diagnose once it reaches an investigation finding or report. The 80/20 principle does not mean 80% verification — it means AI does the mechanical 80% of work and the analyst does the critical 20% of judgment. Verification is in the 20%.
Check it only if the 4th finding seems implausible. If it is consistent with the other 3 findings, it is likely accurate.Incorrect
Consistency with other findings is not evidence of accuracy. AI hallucinations are often internally consistent — Claude may generate 4 findings that form a coherent narrative while the 4th is fabricated. Internal consistency is how AI-generated errors pass surface-level review. The validation discipline checks each claim against source evidence, not against other AI-generated claims.
Verify it only if the 4th finding would change the investigation outcome. If it is supplementary information, the risk of an error is low.Incorrect
Supplementary findings often become critical later. A sign-in that appears incidental during initial triage may become the pivot point when the investigation expands. Categorising findings as critical or supplementary before verification creates a risk that the unverified finding is the one that matters most.
Scenario 6: A security manager wants to deploy Claude across their 8-person SOC team. They plan to give all analysts unrestricted access with no guardrails, arguing that "smart people can figure it out." Design the governance approach you would recommend instead.
Restrict Claude access to senior analysts only and have them review junior analysts' work. This prevents junior analysts from over-relying on AI-generated investigation steps.Incorrect
Restricting access entirely means junior analysts lose the enrichment and documentation benefits that save them significant triage time. Section 0.1 establishes that the value of AI is capability amplification, not capability restriction. The governance solution structures how access is used, not whether access is granted.
Deploy with unrestricted access but add monthly audits to check for quality issues. This balances autonomy with accountability.Incorrect
Monthly audits catch problems after they have been in production for weeks. The damage from an unvalidated AI-generated investigation finding, an incorrect detection rule, or a fabricated attribution in an incident report occurs when it is deployed, not when it is audited. Governance needs to be embedded in the workflow, not appended as an afterthought.
Grant access to all analysts with structured guardrails. Junior analysts use validated prompt templates that enforce the investigation methodology and include verification steps. Senior analysts use open-ended prompts for hypothesis generation. Data handling boundaries from section 0.3 apply to everyone. The validation discipline from section 0.2 is mandatory for all AI-generated artifacts before deployment.Correct
This approach follows the course's governance principle: the difference is not access — it is how the access is structured. Prompt templates encode the methodology, verification steps catch failures, and data handling boundaries prevent sensitive information exposure. Module 7 provides the complete governance framework, and Module 8 covers the deployment playbook with role-specific configurations.
Start with a pilot programme — 2 analysts use Claude for 3 months, then expand based on results. This reduces risk by limiting the initial deployment scope.Incorrect
Pilots are reasonable for untested technology. Claude has documented productivity benefits (EY 2026: 97% of organisations report increased analyst efficiency). The question is not whether to deploy but how to govern the deployment. A pilot delays value for the remaining 6 analysts while providing limited additional information that structured guardrails would address from day one.
Scenario 7: You are investigating an account compromise. You paste a sanitised sign-in log excerpt into Claude that includes your organisation's internal IP ranges (10.x.x.x addresses) and internal domain name (corp.acme-internal.com). Evaluate this data handling decision.
This is safe — internal IP ranges (RFC 1918 addresses) and internal domain names are not sensitive information since they are not routable on the public internet.Incorrect
Internal IP ranges and domain names are low-sensitivity individually but contribute to reconnaissance when combined with other information. An adversary who knows your internal naming convention and subnet structure can craft more targeted phishing, more convincing pretexting, and more effective lateral movement strategies. Section 0.3 places this data in the "requires judgment" category, not the "safe" category.
This falls in the "requires judgment" category from section 0.3. Internal IP ranges and domain names are low-sensitivity individually but could contribute to reconnaissance if combined with other information. The correct approach: replace internal IPs with RFC 5737 documentation addresses (198.51.100.x) and internal domains with placeholders (corp.example.com). The analysis quality is identical with zero data exposure risk.Correct
Section 0.3 establishes three data handling tiers: safe, requires judgment, and never share. Internal network details fall in the judgment tier. The sanitisation takes 30 seconds (find-and-replace) and eliminates the risk entirely without reducing the quality of Claude's analysis. This is the operational discipline that section 0.3 establishes for every AI interaction.
This is a policy violation — internal IP ranges and domain names should never be shared with any AI system under any circumstances.Incorrect
Section 0.3 places internal network details in the "requires judgment" category, not the "never share" category. The "never share" tier is reserved for production credentials, API keys, unredacted PII, and active incident details. Treating all internal information as prohibited would make AI-assisted investigation impractical.
This is acceptable on Claude Team or Enterprise plans because those plans guarantee that data is not used for training.Incorrect
No-training guarantees address one concern (data not being incorporated into future model training) but do not address the broader risk. The data still transits Anthropic's infrastructure during processing. Section 0.3 establishes that AI tools should be treated like any third-party SaaS tool — apply your organisation's vendor data handling policies regardless of plan tier.
Scenario 8: You run a prompt from this course against Claude and get a different recommendation than the course material. The course says to check MfaDetail first when triaging an AiTM alert. Claude says to check the source IP first. Design the correct response to this disagreement.
Follow Claude's recommendation — AI models are trained on more data than any single course and may have a more comprehensive perspective on investigation methodology.Incorrect
AI recommendations are generated from training data patterns, not from production-tested investigation methodology. The course material is written by a practitioner who validated the approach against real incidents. The validation discipline from section 0.2 establishes that when the course and Claude disagree, follow the course material first and then investigate the discrepancy.
Follow the course material without question — the course is the authoritative source and Claude's suggestion should be ignored.Incorrect
Following the course material is correct as the default action, but ignoring the disagreement misses a learning opportunity. Understanding WHY both approaches exist develops the judgment that the course is building. Claude's approach may be valid for a different context — understanding that context deepens your expertise.
Ask Claude which approach has more empirical support and follow whichever recommendation Claude defends with better evidence.Incorrect
Claude will generate a confident defence of whichever position it is asked to support — this is the confident-but-wrong failure mode from section 0.2. Asking AI to arbitrate a disagreement between AI and human expertise is circular reasoning. The evidence comes from your investigation experience and the course's tested methodology, not from AI self-assessment.
Follow the course material, then investigate the disagreement. Ask Claude to explain its reasoning. You may discover Claude's approach is optimised for a different context (environments without MFA, or an era when source IP was a more reliable indicator). The learning is in understanding both approaches and making an informed decision for your environment. Neither source replaces your judgment.Correct
This is the judgment boundary in practice, as described throughout sections 0.1 and 0.2. The course teaches the methodology. Claude accelerates the execution. Neither replaces the practitioner's judgment. Understanding why both approaches exist — and which applies to your specific context — is the skill the course builds across all 11 modules.
How was this module?
Your feedback helps us improve the course. One click is enough — comments are optional.
Thank you — your feedback has been received.
© 2026 Ridgeline Cyber Defence™ Ltd. Content may not be reproduced or redistributed. Worked artifacts may be adapted for use within your organization.