AWS IR Course Toolkit: Evidence Sources, the Query Lab, and the Lab

You have the mental model now. The control plane is the evidence, resources are temporary, identity is the perimeter, and attacks are chains you trace through the log. This section covers the practical side: the evidence you investigate with, the environment you investigate in, and the people you will follow through it.

Your toolkit is five evidence sources

An AWS investigation is built from a small set of log sources, and most of the course is learning to read them and ask the right questions of them. Five carry the weight, each answering a different question.

Each answers a different question, and the art of investigating is reaching for the right one. When you need to know who made a change, CloudTrail has it, because every management action is an API call. When GuardDuty raises a finding, it is telling you its detection logic noticed something, which is a starting point, not a verdict. When you need to know whether a compromised instance actually connected to an external address, VPC Flow Logs hold the answer that CloudTrail does not, because a network connection is not an API call.

When you need to know what a security group or a role looked like at 14:30 yesterday rather than now, Config's state history is where you go. And when the question is which specific objects an attacker read out of a bucket, only S3 data access records can answer it, and only if they were being collected. A surprising amount of weak cloud investigation comes down to asking the right question of the wrong source.

Module 2 takes each apart in detail: what it captures, what it misses, and how long it lasts. For now, hold the shape, because knowing which source answers which question is half of investigating well.

In practice you rarely close a case from one source. You pivot between them, and the pivots are where the picture forms. A GuardDuty finding names an access key; you take that key into CloudTrail and pull its full activity; one of those calls launched an instance, so you go to Config for that instance's configuration and to VPC Flow Logs for what it connected to; an exfiltration call points at a bucket, so you check whether data events recorded what was read.

Each source answers one question and hands you the identifier you need to ask the next one of another source. Learning to move fluidly across them, carrying the access key, the instance ID, the time window from one to the next, is what turns five separate logs into a single account of what happened. The modules build that habit case by case until it is second nature.

Every one of these sources returns evidence as nested JSON, the structure you have already seen in the CloudTrail records throughout this module. The fact you need is rarely a top-level field; it is buried a few levels down, in a userIdentity object or a service.action block. Reading that structure fluently, and pulling out the one field that matters, is a skill in itself, and it is the same skill whether you are looking at a record in the query surface, a finding in GuardDuty, or the output of a CLI command.

The query modules build it deliberately so that by the time a record has five levels of nesting, reaching the field you want is automatic.

You query these through the surface built into this course, introduced in AWS0.2, which runs real SQL against the prepared evidence, plus the AWS Query Lab for free-run practice. In a live account you would reach the same evidence through the AWS Console for a quick look and the AWS CLI for anything programmatic. The first CLI command every responder runs in an unfamiliar account is the one that answers "who am I right now."

The CLI returns structured JSON, the same nested shape as the records you have already read. This is your identity check at the start of any engagement.

$ aws sts get-caller-identity
{
    "UserId": "AIDAEXAMPLEANALYST",
    "Account": "222222222222",
    "Arn": "arn:aws:iam::222222222222:user/analyst.ridgeline"
}

That output says you are operating as an analyst identity in account 222222222222, which is Northgate's security account. Reading nested JSON like this, and pulling the field you need out of it, is a skill the query modules build deliberately, because every source you touch returns it.

The CLI does far more than identity checks; in a live response it is the collection tool. You list the trails configured in an account to confirm what is being logged, describe security groups and roles to capture their current state, pull GuardDuty findings, and gather the configuration history Config holds, all scriptable and repeatable so that a collection step you run today can be re-run identically tomorrow. Confirming what is logged is usually the first thing you check, because it tells you what evidence you can expect to have.

$ aws cloudtrail describe-trails \
    --query 'trailList[].{Name:Name,Bucket:S3BucketName,MultiRegion:IsMultiRegionTrail,Org:IsOrganizationTrail}'
[
    {
        "Name": "ne-org-trail",
        "Bucket": "ne-cloudtrail-logs-222222222222",
        "MultiRegion": true,
        "Org": true
    }
]

That output tells you a great deal in four lines: Northgate runs one organization-wide trail, it covers every region, and it delivers to a bucket in the security account. So management events from all four accounts are being captured and retained durably, which means a timeline reaching back months is achievable. Had this returned an empty list, or a single-region trail, you would know your evidence was far thinner before you spent any time looking for it.

The course shows these collection commands at the point each source becomes relevant, so you finish able to gather evidence in an unfamiliar account, rather than only able to read evidence someone else gathered.

Querying at scale works the same way conceptually whether you are in the course or a real account. The trail's records land as files in S3, and you run SQL against them, which in a production environment usually means Amazon Athena or CloudTrail Lake. The query surface in this course runs real SQL against the prepared evidence for exactly that reason: the queries you write here are the queries you would write against a live trail, so the skill transfers directly rather than being a teaching simplification you have to unlearn. That is the deliberate design choice behind learning on SQL rather than a course-only query language.

The cast you will follow

Northgate's people appear in the logs as IAM principals, and the investigations follow them by name. Knowing the cast in advance means you recognize who is who when their ARN shows up in a record. Marcus Webb, the security architect, works in the production account as m.webb and makes the kind of infrastructure changes you saw in AWS0.3 and AWS0.4, so his activity is the baseline of legitimate administrative work you measure anomalies against. Rachel Okafor is the CISO, Elena Petrova runs governance and risk, Phil Greaves is the IT director, and Tom Ashworth and Priya Sharma are the L1 analysts who catch the first alerts and hand them up.

The identity at the center of the opening incident is m.chen-dev, a developer in the development account whose long-lived access key leaks, the kind of routine, low-drama account that becomes an attacker's way in precisely because nobody is watching it closely.

Two service roles matter early, because services are identities too. app-server-role is attached to the production EC2 instances and is the role whose credentials an attacker steals through the metadata service. cross-account-deploy is the role that bridges the development and production accounts, the kind of trust relationship that lets a foothold in a low-value account reach a high-value one. The attacker is external and holds no identity of their own.

They never appear as a principal, because there is no "attacker" account; they operate entirely by abusing the credentials of the principals above, which is exactly why the investigations are a hunt through identity rather than a search for an intruder's name. You will come to know these identities well enough that an out-of-character action by one of them reads as wrong on sight, which is the instinct that catches real incidents.

The environment you investigate in

All of this lives in the Northgate AWS Organization you met in AWS0.1, four accounts that separate risk and each play a role in the cases.

This central design shapes how you work. You investigate from the security account, where the organization-wide trail collects every account's activity in one place and GuardDuty aggregates findings across the whole organization. That single vantage point is what lets you follow an attacker who starts in development and crosses into production without losing the thread at the account boundary. It is also why the account an event occurred in is evidence in its own right: an action that is routine in the development account may be alarming in production, and the same API call means different things depending on where it happened.

As you work the cases, you will read the account ID in a record the way you read the principal and the source IP, as part of the context that tells you whether what you are looking at is normal. The separation into four accounts is not bureaucratic tidiness; it is a control that contains blast radius and, for you, a source of signal.

Running the lab, in brief

You can take the whole course on the prepared evidence with nothing to set up, which is the path most learners use and the one every lesson is written for. If you want to generate the evidence yourself, the lab pack builds this Organization in your own AWS account with infrastructure-as-code, produces baseline activity, and runs the attacks for you to investigate, at a running cost of a few dollars while it is up.

Building it yourself is worthwhile if you learn best by standing the environment up and breaking it, or if you want to practice the collection commands against live services rather than prepared output. Whichever path you choose, the investigative work is identical: the prepared evidence is drawn from the same Organization the lab pack builds, so the records you query in the lessons are the records the lab would generate.

The full setup walkthrough belongs to Module 1, where you stand the environment up alongside the first real queries, so there is nothing to install yet. For now you have everything you need to start.

Carrying this into Module 1

You arrived at this module with whatever instincts your background gave you. You leave it with a different starting posture for any AWS incident, and it is worth naming the shifts you now carry forward. Four habits make up the whole mental model, and every module from here applies them to a specific class of attack.

The control plane is the crime scene, so you investigate the record of API calls rather than hunting for a breached host. Resources are ephemeral, so you treat the durable log as primary evidence and check retention before you commit to a timeline.

Identity is the perimeter, so you start from the principal and the credential, not the network address. And attacks are chains, so you treat any single finding as one link and pull it in both directions.

That completes the orientation. Module 1, The AWS Detection and IR Landscape, begins the real work: where each piece of evidence lives, the shared responsibility line for incident response, and your first queries against the Northgate environment you just met.