0.2 Determinations and Their Reasons

Introduction

A determination is the assessor's decision on a single determination statement, and it has exactly two possible values. NIST SP 800-53A, the method this course follows, records each statement as satisfied or other than satisfied. Satisfied means the evidence shows the statement holds.

Other than satisfied means anything else, and that "anything else" covers two situations an organization must treat in completely different ways: a control that was shown to fail, and a control nobody could show either way.

This course keeps the two values and adds a required reason to every determination of other than satisfied: deficiency, when the evidence shows the statement does not hold, or cannot determine, when the evidence supplied does not settle it. The figure shows the three outcomes and where each one leads.

It is tempting to want more values. A control that worked for most of a quarter feels like it deserves something better than the same label as a control that never ran. 800-53A deliberately offers no middle value, and the reason is about the reader, not the assessor.

A determination is read by people who cannot see the evidence. A middle value such as partly satisfied tells them almost nothing: partly could mean one exception in a thousand or one success in a thousand, and the label cannot say which. A binary value forces the information that matters into the places where it can be stated precisely.

If a statement is other than satisfied, the workpaper says exactly which cases failed, how many, and why the statement could not be shown to hold. A reader then has a decision they can act on and the detail they need to judge how serious it is.

The binary value also keeps the assessor honest. With a middle value available, uncertain cases drift into it, because it never has to be defended. With only two, every determination of other than satisfied has to name what is wrong, and every satisfied has to be backed by evidence covering the whole statement.

The narrower the statement, the more useful the binary value becomes. "The leaver control is effective" is too broad to decide cleanly. "Each leaver account in the period was disabled within one business day" can be decided, and so can the separate statement about application access. Decomposing a control into statements is how an assessment avoids needing a middle value at all.

Each row can now be decided cleanly, and each can come out differently. The procedure might be well designed, the directory part might fail twice in a quarter, and the application part might be impossible to decide because one application keeps no history.

Three statements give three precise results where one broad statement would have given a single vague one. A good test matrix, which Module 3 builds, is mostly the work of finding the right statements.

Two values also make results comparable. A quarter in which a statement was satisfied and a quarter in which it was not can be set side by side without anyone having to interpret a grade. Trends across assessments depend on that consistency, and grades invented case by case would destroy it.

Satisfied is the stronger claim, and it is the one assessors most often reach too easily. It says the evidence shows the statement holds across the population and period it names. Three conditions have to be met.

The evidence has to cover the whole statement. If the statement is about five systems for every day of a quarter, evidence about four systems, or about two months, does not support it.

The evidence has to be reliable: produced by a system or an independent party rather than asserted by the person whose control is being tested, and complete rather than filtered. And it has to show the condition the statement names, not something next to it.

This workpaper meets all three. The statement names five systems and every day of the quarter, so the population is 460 expected jobs, and the job log holds exactly that many records, every one marked Success. The log is produced by the backup system, not typed by anyone. And it shows the thing the statement asks about: a backup completing each day.

When the population is too large to read in full, satisfied rests on a sample, and the determination has to say so. A sample of 25 changes that all passed supports a conclusion about the population only if the sample was drawn so that every change could have been chosen and the population itself was complete.

Module 5 covers how to draw and defend one. Here the population is small enough that no sample is needed, which is always the stronger position when it is available.

Notice how narrow the conclusion is. The workpaper says backups completed. It does not say the backups can be restored, or restored within four hours, or that they contain the right data.

Those are other statements under other clauses of the same standard, and each needs its own evidence. A satisfied determination is only ever as broad as its statement, which is exactly why the statement has to be written before the evidence is read.

A deficiency is the outcome most people expect from an assessment: the evidence shows that the control did not do what the criteria require. It needs positive evidence of a failure, which is a higher bar than it sounds. An assessor who has not found a record is not yet entitled to say the control failed; they are entitled to say they have not found the record.

The change tickets show it plainly. The policy allows an emergency change to be approved after the fact, within two business days. Of the 6 emergency changes in the quarter, three were approved the next business day.

Three have no approval recorded at all, and the export would show a date if one existed, because the other three carry one. That last point is what makes this a deficiency rather than a gap in the evidence: the export is complete for every ticket, it records approvals when they happen, and for these three it records none.

Deficiencies come in two kinds, and the statement decides which one you have found. A design deficiency means the control could not meet the criteria even if everyone followed it: a policy that allows emergency changes to go to the next monthly board cannot meet a two-day limit.

An operating deficiency means the control is designed to meet the criteria and was not followed: the policy here says two days, three changes never reached the board, and the problem lies in what happened rather than what was written. The two lead to different fixes, so the workpaper records which one the evidence shows.

A deficiency always names the cases. "Emergency changes are not consistently approved" is a summary; CHG-30114, CHG-30127, CHG-30138 are facts the change owner can look up, and they are the starting point for the finding in Module 9. The count belongs in the determination as well, because three of six is a different problem from three of three hundred, even though both are other than satisfied.

A deficiency is a statement about the control, never about the people running it. The two late leavers may have been handled by a capable team on a bad week. The determination records what happened to the accounts; the reasons why belong to the finding, once the owner has had the chance to explain them.

The second reason is the one assessors are most often pushed to avoid. Cannot determine means the evidence supplied does not show whether the statement holds or not. It is a legitimate, precise outcome, and forcing a pass or a fail where the evidence cannot support one is a failure of the assessment, not a way of completing it.

The standard says business-critical systems can be restored within four hours. The only restore test in the quarter was of a file server, and it took more than nine hours, which is a deficiency for that server.

For the ERP there is no test at all. The daily backups are tempting evidence, but they show a different thing: data was copied. They say nothing about how long a restore takes, whether the restored system starts, or whether the backup contains what is needed.

Evidence goes missing for different reasons, and the reason shapes the request that follows. The record may never have been produced because the activity never happened. It may have been produced and not kept. It may exist and not have been requested, or been requested and not supplied.

The assessor cannot always tell which, but they record what they asked for, when, and what came back, because a reader will judge a cannot-determine very differently depending on whether the organization looked and found nothing or never looked.

A request written this precisely can only be answered in two ways, and either answer settles something. A record of a test decides the statement. A confirmation that none was done turns the cannot-determine into a confirmed gap in testing, which is a deficiency against clause 2 of the same standard.

So the statement about the ERP cannot be decided. It would be wrong to call it satisfied because backups ran, and it would be wrong to call it a deficiency, because there is no evidence the ERP cannot be restored in four hours; there is simply no evidence either way.

Cannot determine is the honest answer, and it is not a soft one. For a business-critical system with a stated recovery objective, not knowing whether it can be met is itself a serious matter, and the report will say so.

The two reasons for other than satisfied can look similar on a first read, because both start from a statement that could not be shown to hold. The question that separates them is what the evidence actually shows. If it shows a failure, it is a deficiency. If it shows nothing, or too little, or cannot be trusted, it is cannot determine.

Unreliable evidence is the case that trips assessors most often. A leaver ticket marked closed, with access removed, looks like evidence that access was removed. If one of those tickets was closed before the directory shows the account disabled, the ticket is not evidence of removal at all, only of someone closing a ticket.

Where the directory export is available, the assessor uses it and sets the ticket aside. Where it is not, the statement about removal cannot be determined from tickets alone, however many of them say the right thing.

Contradictory evidence needs the same care. The infrastructure manager may say in interview that restores are tested every quarter for every critical system. The restore test export shows one test, of one server.

Interview evidence does not outweigh a system record of the same thing; it tells the assessor what to look for. Here the records contradict the interview, the contradiction is itself recorded, and the determination rests on the records, with the interview noted as uncorroborated.

A cannot-determine outcome often turns into one of the others once better evidence arrives. That is a feature: it tells the organization exactly what record would settle the question. A deficiency stays a deficiency whatever else arrives, unless the evidence it rests on turns out to be wrong.

A practical test separates the two when the evidence is mixed. Ask whether any record that could still arrive would make the statement hold. If none could, because the records already show a failure, the reason is deficiency. If one could, the reason is cannot determine, until that record has been asked for and the answer recorded.

The hardest determinations to write are the ones where a control works almost everywhere. A control that covers 96.5 percent of devices is a good control with a gap, and every instinct says the determination should reflect that. It should, but in the reason and the detail, not in the value.

Each row is other than satisfied, and each is a deficiency.

The percentages in the second column are true and useful. They belong in the workpaper's result and in the finding, where they tell a reader how large the problem is. They do not belong in place of the determination, because the criterion in each case was written as a requirement for every device, every change or every leaver, and the evidence shows the requirement was not met.

A compensating control does not change the value either. The organization might argue that the 31 devices without an EDR sensor are covered by network monitoring.

That may be true and may matter a great deal to the severity of the finding, but it is a different control answering a different criterion. The statement about EDR coverage is decided on EDR coverage, and the compensating control is assessed, and credited, where the finding's risk is weighed.

There is one legitimate route to a different answer, and it runs through the criteria rather than the evidence. If a standard says "at least 95 percent of devices", then 96.5 percent satisfies it. An assessor never adjusts the threshold to fit the result; they apply the threshold the criteria state, and if the criteria state none, they test against the requirement as written and say so.

Partial results are where pressure to soften usually appears. Twelve of fourteen sounds like a good result, and in many senses it is. But the statement asked about each leaver, and two of them did not meet it. The extent goes into the result line, where a reader can see it; the value stays other than satisfied.

The reason is not a footnote to the determination. It decides who does what next, and getting it wrong sends the organization to the wrong work.

A deficiency goes to the control owner as a finding. They need to understand what failed, fix it, and prove the fix with evidence from a later period, which is the remediation and retest cycle of Module 10.

A cannot-determine goes to whoever holds the missing evidence, as a specific request: a restore test record for the ERP from the quarter, or confirmation that none exists. If the evidence arrives, the statement is decided. If it does not, the report records a scope limitation: this statement could not be assessed, and here is why.

Timing puts pressure on this choice. Near a reporting deadline, a cannot-determine looks like unfinished work, and there is a pull to convert it into whichever determination the available evidence leans toward.

The right response is the opposite: state the outcome as it stands, name the evidence still outstanding, and let the report carry the limitation. A report can be updated when the evidence arrives; a determination reached without it cannot be trusted afterwards.

The two can be confused in both directions, with real cost. Calling a cannot-determine a deficiency sends the control owner to fix a control that may work, and damages trust in the assessment when they show it does. Calling a deficiency a cannot-determine hides a failure behind a request for more paper. Neither error is caught by anyone downstream, because nobody downstream sees the evidence.

The report keeps them apart in its structure as well as its words. Deficiencies appear as findings, each with a condition, a criterion and an owner. Cannot-determine outcomes appear as limitations, each naming the statement, the record requested, when it was asked for and the reply. A reader can then see at a glance which problems are known failures and which are gaps in what the organization could show.

Every exercise page in this course ends on determinations, and so does the exam. They are marked on the things this lesson has described, in this order: the value, the reason, the evidence it rests on, and the scope it states.

The reason carries the same weight as the value. An answer of other than satisfied for the ERP restore statement with the reason deficiency gets the value right and the outcome wrong: it would send the infrastructure team to fix a recovery capability nobody has shown to be broken, while the actual problem, the absence of any test, goes unaddressed. The exam treats that as a wrong answer.

The practice tools embedded in the lessons work the same way. Each one gives you records, asks for a determination or a reading of the records, and shows its reasoning once you commit, so you can check two things at once: whether you reached the right value, and whether you reached it for the right reason.

Treat every one as a short assessment rather than a quiz: read the records before the question, and decide before you look at the options.

You will also meet statements where the evidence pack is designed so that cannot determine is the correct answer. Those questions reward the discipline of stopping at what the evidence shows. Writing satisfied because a control looks well run, or deficiency because something seems off, scores nothing there, however plausible it sounds.

The evidence and scope marks are earned in the wording. A determination that cites "the directory export" loses marks that one citing identity/directory-leaver-accounts.csv, and the rows it relied on, would earn. A scope written as "leavers" loses marks that one written as "the 14 leavers HR recorded in Q3" would earn, because only the second tells a reader exactly what was tested and what was not.

Then take one control you know, at work or from a past audit, and ask which outcome its last review really supported. If the review said effective, list the evidence that would have been needed for satisfied, and whether it was actually seen.

The next lesson opens the evidence pack itself: what an organization hands an assessor, and how to read each kind of record before deciding what it proves.