0.3 The Evidence Pack

Introduction

An evidence pack is the set of records an organization supplies so that an assessor can decide determination statements: its standards, the exports its systems produce, the reports it already relies on, and the notes from interviews with the people who run its controls.

Every exercise and the exam in this course give you one. The pack for the third quarter of 2026 holds 54 files about a fictional engineering company's identity, operational and supplier controls, and you can download it whole from the course's resources.

The pack is deliberately imperfect, the way real ones are. Some records are complete, some are filtered, some summarize other records inaccurately, and some say nothing at all. Reading it well is a skill in its own right, and it starts with knowing what kind of material each file is. The figure shows the four kinds and how they relate.

Evidence arrives in response to a request, and the request shapes what arrives. An organization asked for "evidence that leavers are disabled" will send what it believes shows that: perhaps a policy, a screenshot of a disabled account, and a handful of closed tickets. Every item may be genuine, and together they may prove nothing, because the selection was made by the people whose control is being assessed.

The defense is to ask for populations and records, not conclusions. A population is the complete list of things a control should have acted on: every leaver in the period, every emergency change, every server in the inventory.

Activity records show what happened to each member of the population. With both, the assessor can test the control themselves; with only the organization's selection, they can only confirm that the selection says what the organization hoped.

NIST SP 800-53A describes what an assessor examines in terms of assessment objects: specifications such as policies and procedures, mechanisms such as systems and their configurations, activities such as the work people do, and the individuals who do it. A good first request covers all four for every control in scope, and names the period it wants records for. Later requests follow the gaps the first one exposes.

Ask the system, not only the person. Where an export can be produced by someone other than the control owner, a system administrator running a saved query or a read-only account the assessor is given, the evidence is stronger, because nobody with an interest in the result has handled it on the way.

Where only the control owner can produce it, record that, and plan to corroborate it against a second source.

The pack in this course is the answer to a request like that, already received. That spares you the waiting, but it does not spare you the judgment: you still have to decide, file by file, what each item can and cannot show.

The pack is organized by the kind of control each file concerns, not by the kind of evidence it is. A folder of identity records sits beside a folder of operational records, and inside each you will find exports, reports and documents mixed together, which is exactly how material arrives from a real organization.

Download the whole pack as one file, evidence-pack-q3.zip, and unzip it into a folder of its own. Every lesson and exercise refers to files by their path inside the pack, such as identity/hr-leavers.csv.

Two files at the top level help you navigate. The README states the assessment period, the first and last day for which records were requested. The index lists every file with its type and, for each CSV, the number of data rows it holds.

Use the index as a first check on completeness: an export of the quarter's leavers with three rows, or a backup log with a number of rows that does not match the number of systems times the number of days, is telling you something before you open it.

File names describe content, and they are the organization's names, not yours. A file called mfa-dashboard.json is a dashboard; it is not, by being in the pack, evidence that MFA is enforced. Read every name as a claim about the file, to be checked like any other claim.

The README's last sentence is worth taking literally: nothing in the pack has been checked for you.

The lessons work through parts of it as worked examples, and the exercises ask you to decide statements from it, but neither tells you in advance which files are reliable. Each lesson's demonstration uses some files and leaves others for practice, so a file you have not yet read may still be the one that settles a statement.

When you work through an assessment, keep the pack itself unchanged. Copy what you need into your own working files and annotate those. The original files are the evidence; once they have been edited, nobody can tell what was received and what was added.

The layout is also a reminder of what was not supplied. A pack with no folder for physical security or for application changes tells the assessor something about scope before a single file is opened.

A quick way to use that reminder is to list, before opening any file, each control the criteria documents cover and the folder its records should be in.

A control with no records in the pack, or a folder of records that no criterion asks about, is a question for the plan in Module 2. It is far cheaper to settle at that stage than after a determination has been drafted on whichever files happened to be there.

The criteria folder holds the organization's own standards and policies. Each control in the pack is measured against one of them, and the first thing to record about a criteria document is exactly which version you are using.

The version and approval date matter for a reason that is easy to overlook. A standard approved partway through the assessment period imposes its requirements only from that date; for the weeks before, the previous version applies.

Every standard here was approved before 1 July 2026, so each applies for the whole quarter, but that is something to check rather than assume. An assessment that applies a requirement before it existed reports a failure that was not one.

Read each standard twice: once for its requirements and once for its definitions. The Access Control Standard defines the leaver limit from "the leave date HR records", which settles which date counts. The Backup and Recovery Standard names exactly which systems are business-critical, which settles the population for every backup statement.

Definitions like these are criteria too. They decide arguments before the evidence is read, and an assessment that ignores them will find itself having those arguments at the end, with the control owner, in front of a reader.

Organizations often map their standards to an external framework, clause by clause, and quote the mapping in the standard itself. A mapping helps an assessor say which framework requirement a determination supports. It does not change what the organization's own clause requires, and where the two differ, the assessment says which one it tested against.

Where a standard is silent, note the silence. Several of these standards require an alert or a review without saying how quickly it must happen, and a statement about timeliness cannot be tested against a requirement that does not exist.

Most of the pack is CSV exports, the format systems produce when someone asks for their records. Each one in this pack starts with a fixed label line saying the data is fictional training data, then a header row naming the columns, then one row per record.

# FICTIONAL TRAINING DATA. Northgate Engineering is fictional. Ridgeline Cyber GRC301
change,type,summary,implemented,cabApproved
CHG-30100,Normal,Certificate renewal,2026-07-02,2026-06-29
CHG-30101,Normal,Application release,2026-07-03,2026-06-30
CHG-30102,Normal,Application release,2026-07-06,2026-07-03

Read the header before the rows. It tells you what the system recorded, and what it did not.

The change export records the type of change, the date it was implemented and the date it was approved. It does not record who approved it, or whether the approval came before or after the board met. A statement about who approved emergency changes cannot be decided from this file, whatever its rows say.

Check the units of every date and time. Dates such as 2026-07-02 carry no time zone and are dates in the organization's own calendar. Times such as 2026-07-06T10:15:00Z end in Z, which means UTC; the organization works in UK time, an hour ahead of UTC in summer.

Two exports that record the same event in different zones will disagree by an hour, and near midnight that can move an event to a different day.

Be careful how you open a CSV. Spreadsheet applications guess the type of each column and convert what they guess: dates are reformatted to the local convention, leading zeros disappear from IDs, and long numbers turn into scientific notation.

Opening an export by double-clicking it can change the values you are about to rely on. Import it with every column as text, or read it with a tool that does not convert, and keep the original file beside your working copy.

Finally, check the rows against the index. If the index says 40 rows and the file holds 40, the file is as supplied. If the counts differ, something was changed after the pack was assembled, and the file cannot be relied on until you know what.

Some files in the pack are summaries: dashboards, SLA reports, review summaries. They are useful because they say what the organization believes, and dangerous because they look like evidence of the thing they summarize. A summary is only as good as the population and the logic behind it, and neither is visible in the summary itself.

{
  "title": "Identity security dashboard: MFA coverage",
  "generated": "2026-09-30T07:00:00Z",
  "tiles": [
    {
      "name": "MFA coverage",
      "value": "98.0%",
      "detail": "785 of 801 users"
    }
  ]
}

This is the identity team's MFA dashboard as it stood at the end of the quarter. It reports 98.0% coverage, and the detail line gives the numerator and denominator: 785 of 801 users. The registration export beneath it holds the underlying records:

userPrincipalName,userType,isMfaRegistered,methodsRegistered
r.okafor@ne.com,member,True,passKeyDeviceBound
p.greaves@ne.com,member,True,microsoftAuthenticatorPush
m.webb@ne.com,member,True,mobilePhone
e.petrova@ne.com,member,True,passKeyDeviceBound

Note when the summary was produced, too. The dashboard was generated at 2026-09-30 07:00 UTC, the morning of the last day of the quarter. Anything that changed later that day is not in it, and a statement about the state at the end of the period needs a record taken after the period ended.

Two questions turn the dashboard from a conclusion into a lead. Which accounts does the denominator include? The registration export holds 814 accounts, of which 801 are ordinary members, 8 are guests and 5 are service accounts, and the dashboard's 801 matches the members alone. And what does the numerator count?

The column is isMfaRegistered, which records that a method is registered, not that the account is required to use it at sign-in. The dashboard is accurate about what it counts. Whether it counts the thing the criterion asks about is a separate question, and it is the subject of Module 1's exercise.

Summaries are not worthless. They show what the organization believes about its own controls, and they often show what management is being told. When a summary and its records disagree, that gap is itself useful evidence, because it suggests someone is relying on a figure that does not mean what they think it means.

The interviews folder holds notes from conversations with three control owners. Interviews are one of the three assessment methods 800-53A names, and they serve a specific purpose: they tell the assessor how a control is meant to work, who does what, and where to look for records. They are rarely enough on their own to decide whether it worked.

Read this as a set of claims to test. The infrastructure manager says restores are tested every quarter for all the critical systems, and come in well under four hours. Each part of that sentence points to a record that should exist: a test record per critical system per quarter, with start and completion times.

If the records exist and agree, the interview has been corroborated. If they do not, the interview has been contradicted, and the contradiction is itself something the report records.

Interview notes also carry information no export holds. Who performs a control, how they learn they need to, what they do when the system is down: all of this shapes the design of the control, and it is how an assessor learns which records to request next.

Treat an interview as the most useful map of where the evidence is, and as the weakest evidence of what the evidence will show.

Record every interview the same way: who was interviewed, their role, the date, the questions asked and the answers given, in notes the interviewee has had a chance to correct. Notes nobody has confirmed are the assessor's recollection, and a control owner who later disputes a finding will dispute the notes first.

Never let an interview fill a gap in the records. A confident statement that "nobody touches it by hand any more" is not evidence that the leaver process is automated, and it is especially not evidence when the directory shows two accounts disabled days late.

Two exports in the pack have a header and no rows. An empty file is a result, and it is easy to read too much into it in either direction.

An export with no rows shows that the query behind it returned nothing. It does not show why.

The SIEM silence alert export might be empty because no source went silent, because sources went silent and no alert fired, because alerts fired and were filtered out of the export, or because no such alert was ever configured. Each explanation leads to a different determination, and the file alone cannot choose between them.

Here, the daily ingest records show sources that did go silent, which rules out the first explanation and leaves the rest open.

When an export comes back empty, ask for the query or report definition that produced it, and the date it ran. A query that searched for tickets mentioning "tabletop" will not find tickets that say "TTX" or "exercise". With the definition in hand you can judge whether empty means nothing happened or only that the search did not find it.

Missing files are subtler, because nothing reminds you they are missing. The pack has no record of anyone reviewing the MFA exclusion group, no restore test of the ERP, and no bridge letter from the payroll supplier. You only notice those by starting from the criteria and asking, for each requirement, which record would show it was met, then looking for that record.

That is the habit this lesson exists to build. Start from the statement, name the record that would decide it, then look for the record. Starting from the files and asking what each one proves will always miss what is not there.

Register an empty file like any other item, with its header, its zero rows and, once it arrives, the query that produced it. An empty export left out of the register is easy to forget when the workpaper is written, and with it the question of what the emptiness means.

An assessment's conclusions are only as defensible as the link between them and the evidence. If someone later asks what the leaver determination rested on, the answer has to be specific: this file, received on this date, in this form. The simplest way to fix that form is a cryptographic hash.

A hash is a short fingerprint calculated from a file's exact contents. The same file always produces the same hash, and any change to the file, even a single character, produces a different one.

Recording the hash of each file when you receive it lets anyone confirm later that the file you relied on is the one you were given. On Windows, Get-FileHash in PowerShell calculates a SHA-256 hash; on macOS and Linux, shasum or sha256sum does the same.

Screenshots deserve the same treatment and more caution. They show one screen at one moment, they are easy to crop or stage, and they often omit the date and the system they came from.

Where a screenshot is the only evidence available, record what it shows, when and from where it was taken, and hash it like any other file; where an export of the same data exists, prefer the export.

Keep a log of every request, too: what was asked for, of whom, when, and when it arrived. The log is how the report later shows which evidence came late or never came, and it settles disputes about what the organization was asked to supply.

Hashes do not tell you a file is accurate. A filtered export hashes as faithfully as a complete one. They tell you it has not changed since you received it, which is the property you need when your workpapers cite it.

The evidence register you will meet in the next lesson has a place for exactly this: what each item is, who supplied it, when, what period and population it covers, and how far it can be relied on. Building that register is the first piece of real work in any assessment, and it is this lesson's practice.

Then think of the last time your own team supplied evidence to an auditor or a customer. Which items were system exports, which were summaries, and which were selected by the people who own the control? The answer is usually a good indication of how much the resulting report could be relied on.

The next lesson introduces the seven templates this course uses, from the assessment plan to the retest record, and how each one carries an assessment from scope to closure.