0.6 GuardDuty and Security Hub Across an Organization

Introduction

The trail is the record; detection is what tells someone to read it, and when. In AWS that is GuardDuty, which analyzes CloudTrail, VPC flow logs and DNS logs for signs of attack, and Security Hub, which gathers GuardDuty's findings with its own configuration checks in one place.

Beside them sits AWS Config, which records how resources were configured over time, the history an investigation needs to restore what an attacker changed. In an organization, all three should be run from one account for every account, so that no member can switch them off and nothing is missed because someone forgot to enable a service.

Detection is also where readiness and response meet most directly, and where the course's period is most instructive. A finding is the moment an organization first knows something is wrong, and everything after it, triage, declaration, containment, depends on that moment arriving early and being acted on.

The course's investigations show both sides: GuardDuty raised findings within minutes for most of the period's attacks, and nobody acted on any of them for days. This lesson audits how Northgate does it: the accounts that produce findings, what Security Hub gathers, where Config keeps history, which GuardDuty features match the workloads Northgate runs, how long findings survive, and whether anyone acts on them.

The figure is the arrangement AWS recommends and Northgate follows: the security account is the delegated administrator, enabling detection in every member and receiving every finding. Config is the exception, recording per account, which is where Northgate's gaps begin.

The audit in this lesson uses a summary a readiness review would assemble, gathered from several API calls in the security account, and the queries read what those services produced during the period. Configuration says what detection could see; the record shows what it did see and what happened next.

Delegated administration is the arrangement that makes detection an organization's rather than each account's. The management account designates one member, here the security account, to run GuardDuty and Security Hub for everyone; that account enables detectors, sets features and receives every finding. The query counts findings by the account that raised them.

SELECT accountid, count(*) AS findings, count(DISTINCT type) AS finding_types
FROM guardduty_findings
GROUP BY 1
ORDER BY 1

The query groups GuardDuty's findings by the account each concerns, with how many kinds of finding each raised. Findings came from dev and prod, where the attacks were, and none from management or security. That is not a coverage gap: the delegated administrator enables a detector in every account, including ones that later produce nothing.

Delegation also means one team manages detection for the whole organization: enabling features, setting suppression rules, exporting findings. What delegation buys is that a member account cannot disable its own detector, as the backdoor user found on 18 May when its attempts to delete and change the dev detector were refused, and that every finding arrives in one place.

The refused attempts to delete and change the dev detector are in the record because the trail records them in dev; GuardDuty's own administration happens in the security account, where those calls would have had to be made to succeed.

That is the same pattern as the organization trail in 0.3: the control lives in an account the workload accounts cannot reach, and attempts against it from below are refused and recorded.

The record also shows the arrangement working from the analyst's side. GuardDuty's findings carry the account they concern, so an analyst in the security account can see in one list that the leaked CI key and the escalation were in dev and the backup theft and stolen instance credentials in prod, without signing in to either.

That single view across accounts is what makes it possible to see, as Module 10 did, that several incidents might be connected.

Security Hub is where Northgate gathers what its detection produces, from GuardDuty and from its own checks. The query counts its findings by the product that raised them.

SELECT productname, count(*) AS findings
FROM securityhub_findings_asff
GROUP BY 1

Eleven GuardDuty findings and four results of Security Hub's own EC2.8 check, the IMDSv2 requirement, one per instance. Security Hub runs many more checks than that against a standard such as AWS Foundational Security Best Practices; the sample period keeps only the one the course's investigations needed.

Security Hub's role is aggregation: one queue for every source of findings in every account, in a standard format, with a workflow status that records whether anyone has acted. The four control results are a reminder that Security Hub also checks configuration against a standard, and the one that failed, app-prod-01's metadata setting, named the weakness Module 7's attack used five months before it was used.

Aggregation only helps if the queue is actually worked. Security Hub records a workflow status on every finding, new, notified, suppressed or resolved, and that status is how an organization shows, to itself and anyone who asks, that findings are being handled. Northgate's twelve untriaged findings all still read new at the end of the period, which section 7 takes up.

Security Hub can also gather findings from other AWS services and from many third-party tools, and its standards check every account against a consistent set of controls. For a small organization the value is mostly the single queue and the workflow status; for a larger one, the consistent checks across hundreds of accounts matter as much.

Security Hub's control results deserve exactly the same ownership as GuardDuty's threat findings. A failed check is a statement that a resource is configured in a way an attacker could use, and the one in Northgate's record was used. A readiness review should count failed controls with no owner, as it counts untriaged threats.

AWS Config is the third service, and the one where Northgate's coverage is thinnest, because it is configured account by account rather than once for the organization. The query counts Config's change items by account and resource type.

SELECT ci.awsaccountid AS account, ci.resourcetype, count(*) AS change_items
FROM awsconfig
CROSS JOIN UNNEST(configurationitems) AS t(ci)
WHERE configsnapshotid IS NULL
GROUP BY 1, 2
ORDER BY 1, 2

The query counts the configuration items Config recorded for changes, by account and type. Configuration history exists only in dev, and only for instances and three IAM types. Prod's recorder records instances alone, and no account records S3 buckets.

The consequence appeared in Module 10: dev's role trust and policy version could be restored exactly from Config, while the prod backup bucket's settings had to be restored from what its owner wrote down.

Config is the source an investigation reaches for, first and often only, when it needs to know what something looked like before an attacker changed it, and it can only answer for what its recorder was told to record.

Config's coverage is usually a cost decision too, since it charges per configuration item recorded, and recording everything in a busy account can be expensive. But IAM and S3 change rarely and matter enormously in an investigation, so recording them costs little and buys exactly the history an attacker's changes need.

Northgate's prod recorder was set to instances only, and whatever the original reason, nobody revisited it when the question changed from what runs to what changed.

Config can also be aggregated across an organization: an aggregator in the security account collects configuration from every account and Region, so an investigator can search history centrally rather than account by account. Northgate has none. The gap matters less than the recording gap, because an aggregator can only gather what recorders record.

Config's recording gap and the trail's data event gaps of 0.4 compound each other. Where the trail records the change and Config records the before-state, an investigation knows exactly what was changed and to what; where only the trail records it, the investigation knows the new state from the request and has to reconstruct the old one from older requests, if they are still kept.

The three services' configuration is gathered into one view for the audit, as a readiness review would assemble it.

The auditor shows a readiness review's summary of the three services' organization settings, each line as the relevant API reports it. Four gaps. GuardDuty findings are never exported, so they vanish after 90 days. Runtime Monitoring is off, so nothing watches what happens inside an instance.

Lambda Protection is off, so functions' network activity goes unanalyzed, and with no Lambda data events in the trail, a function's behavior is close to invisible. And prod's Config recorder records instances only. Two lines that look like gaps are not, and an audit that flagged them would waste effort: EKS audit logs are off because Northgate runs no EKS clusters, and auto-enable for all members is exactly right.

The audit also has a sequence. Exporting findings is the cheapest and most urgent: it costs storage only and starts protecting the record at once. Prod's Config recorder follows, because every day without it is a day of IAM and bucket changes with no before-state.

Lambda Protection is a single setting. Runtime Monitoring takes the most work, because it relies on an agent on each instance, and is best rolled out to internet-facing instances first.

None of the four would have prevented an attack in the period. Each would have changed what an investigation could establish: Runtime Monitoring what happened on app-prod-01 after its credentials were fetched, Lambda Protection whether the staged function ever reached out, the export whether a finding could be read in a year, and prod's Config history what the backup bucket looked like before 16 May.

Features are also where cost enters detection, and the decisions should be explicit. Runtime Monitoring and Lambda Protection are priced by the volume they analyze, and an organization with many instances may enable Runtime Monitoring selectively. That is a reasonable choice when it is a choice; the audit's question is whether each feature that is off is off because someone decided, or because nobody did.

The four gaps also have owners outside the security team in two cases. Runtime Monitoring needs an agent on each instance, which the teams running those instances must accept; prod's Config recorder belongs to whoever administers the prod account. A readiness review that names the gap without naming who must close it has done half its work.

GuardDuty's foundational analysis covers CloudTrail management events, VPC flow logs and DNS logs in every account. Its other protections are features, each tied to a kind of workload, and each worth enabling only where the workload exists. The gate below is the test.

AutoEnableOrganizationMembers ALL
Features for EC2 and Lambda
PublishingDestinations
an owner for each finding

The gate's four clauses are the audit in miniature: on everywhere, matched to workloads, output kept, someone responsible.

Northgate runs EC2 instances and Lambda functions, and it has S3 protection on, which analyzes data events for suspicious access. It has nothing for the inside of instances or for Lambda's network activity. Both matter here: app-prod-01 is internet-facing and was reached through its portal, and a function staged by an attacker would run with a role's permissions. EKS and RDS protections are rightly off, because Northgate runs neither.

Features are not the only way GuardDuty's coverage can be weaker than it looks. Suppression rules archive findings before anyone sees them, and trusted IP lists stop findings for listed addresses being generated at all; 9.4 audits both.

Northgate has neither, so every finding GuardDuty could raise reached the security account. An organization auditing its own detection should read its suppression rules and lists with the same care as its features.

The match between features and workloads also changes over time. An organization that adopts containers on EKS, or moves a database to RDS, needs the matching protection enabled when the workload arrives, not after its first incident. Reading the organization configuration whenever a new kind of workload is introduced is the cheapest way to keep detection matched to what actually runs.

Each of those is also something Module 10's response had to work around. The staged function was proven never to have run only by an indirect route; app-prod-01's memory had to be captured because nothing else would say what ran on it. Detection features are cheaper than those workarounds, and far faster.

Findings have the same retention problem as logs, and it is easier to miss because nothing deletes them visibly. The query works out when the first of Northgate's findings expires.

SELECT min(createdat) AS first_finding,
  date_add('day', 90, from_iso8601_timestamp(min(createdat))) AS first_expires,
  max(createdat) AS last_finding
FROM guardduty_findings

The first finding, from 11 May, leaves GuardDuty on 9 August, and the rest follow within a week. GuardDuty keeps a finding for 90 days. After that it exists only in Security Hub's copy, for as long as Security Hub keeps it, and in nothing Northgate controls.

A publishing destination exports every finding to an S3 bucket as it is raised, where it can be kept with the rest of the record, protected the same way, and queried in Athena beside CloudTrail. Northgate has none. The course reads its findings from a table that exists because the course needed it; Northgate itself would lose the first of them in early August.

Security Hub's copy of each finding is not a substitute for an export. It has its own retention, its findings can be archived or updated by automation, and its format is not the one GuardDuty produced; the export is the original, kept on the organization's terms.

Exported findings can also be read beside CloudTrail in Athena, as the course reads them, which makes them part of the record rather than a separate console.

Exporting findings is a single setting made from the delegated administrator account, and it applies to every account's findings. The bucket it writes to should be treated as part of the log archive: in the security account, versioned, with Object Lock, and readable by the same investigation role as the trail, so the findings survive as long as the record they describe.

With the export in place, every future finding becomes a lasting part of the record the course's methods read.

The last part of the audit is not a setting at all. The query counts how often the security team read findings in the period.

SELECT eventsource, eventname, count(*) AS calls,
  count(DISTINCT element_at(split(useridentity.arn, '/'), 3)) AS people
FROM cloudtrail_logs
WHERE recipientaccountid = '100000000102'
  AND eventsource IN ('guardduty.amazonaws.com', 'securityhub.amazonaws.com', 'config.amazonaws.com')
GROUP BY 1, 2

The query counts the security team's calls to the three services in the security account. Three people in the security account listed GuardDuty findings 132 times and read Security Hub findings 79 times in five weeks. No finding was ever acknowledged, assigned or resolved. Detection was configured, findings were raised within minutes of most attacks, and people looked at them.

What was missing was a rule that each finding has an owner who must act by a deadline. 10.8 found that gap as the main cause of the delay between detection and response; a readiness review can find it before any incident, by asking who owns each finding and checking Security Hub's workflow status.

Ownership has a concrete form, and it needs no new tool. Each finding is assigned, by account or by type, to a person or team; each owner has a deadline set by severity; Security Hub's workflow status records each step; and a finding still new past its deadline is escalated to someone who can make it move.

None of this needs a new tool. It needs a decision, written down, and a weekly look at how many findings are still new, which is the query 10.8's review ends with.

The readers in the record were doing something useful: they knew the findings existed. But reading without recording a decision leaves no trace that the finding was considered, and no way for anyone else to know whether it was. The workflow status is how a team's judgment becomes part of the record.

The ownership gap also explains why the detections Module 9 builds would not have helped on their own. A detection that fires earlier than GuardDuty produces an alert sooner; if the alert lands in the same unowned queue, it is read sooner and acted on no sooner. Faster detection and clear ownership are a pair, and a readiness review that improves one without the other improves neither.

Northgate's three readers were the right people; the gap was that no one of them owned a finding until it was closed, and so each could reasonably assume another was handling it.

The steps below audit detection across any AWS organization, from the delegated administrator's account.

aws guardduty list-organization-admin-accounts

aws guardduty describe-organization-configuration --detector-id <id>

The procedure's last step is the one most readiness reviews skip, because it is about people and process rather than configuration, and it is the one Northgate's period shows mattered most.

Applied to Northgate, the four steps confirm one delegated administrator for GuardDuty and Security Hub, find two features missing for workloads Northgate runs, find no export of findings and Config history only in dev, and find no owner for any finding. Each is a configuration change except the last, which is a decision about people. The footer is the measure configuration cannot give, which Module 9 supplies.

Practice

Three questions about detection.

Next, 0.7 The Investigator's Access audits who can respond, with what access, and where the evidence would go.