Measuring Identity Security Posture

Why metrics, not features

Identity security is often described in terms of features deployed: "We enabled MFA." "We configured Conditional Access." "We deployed PIM." Features are binary — enabled or not enabled. They tell you nothing about coverage, effectiveness, or gaps. And they don't answer the question your CISO will actually ask: "How secure are we?"

Metrics are continuous. They tell you how much of your environment is protected, how well the protection is working, and where the gaps are. A feature list that says "MFA enabled" is not the same as a metric that says "62% of sign-ins use phishing-resistant methods, 34% use push notification, and 4% use SMS." The feature is the same in both cases. The security posture is completely different — the 34% using push are vulnerable to AiTM, and the 4% on SMS are vulnerable to both AiTM and SIM swapping.

Metrics also make the investment case. When you deploy phishing-resistant authentication (EI2) and the resistant percentage moves from 3% to 45% in a quarter, that's a measurable improvement you can present to leadership. When the CA evaluation rate goes from 73% to 98%, you've eliminated 25 percentage points of implicit trust surface. These numbers justify continued investment in ways that "we enabled another feature" never will.

Four metrics capture the state of an identity security program. Each maps to a specific area of the course, a specific set of attack patterns, and a specific set of controls. Together, they give you a dashboard that answers the CISO's question with numbers instead of feature names.

Metric 1: Phishing-resistant authentication coverage

This is the percentage of successful sign-ins that used phishing-resistant authentication methods — FIDO2 security keys, passkeys (device-bound or synced), or certificate-based authentication. It directly measures your exposure to AiTM credential phishing and MFA fatigue, the two most damaging identity attack techniques in production environments today.

The calculation is straightforward: divide the number of sign-ins that used a phishing-resistant method by the total number of successful sign-ins. The result is a single number that captures the most important dimension of your authentication security posture.

// Metric 1: Phishing-resistant authentication coverage
SigninLogs
| where TimeGenerated > ago(30d)
| where ResultType == "0"
| extend AuthMethod = tostring(AuthenticationDetails[0].authenticationMethod)
| extend IsPhishingResistant = AuthMethod in ("FIDO2 security key", "Passkey (Microsoft Authenticator)", "X.509 Certificate")
| summarize
    TotalSignIns = count(),
    PhishingResistant = countif(IsPhishingResistant),
    Phishable = countif(not(IsPhishingResistant))
| extend ResistantPercent = round(100.0 * PhishingResistant / TotalSignIns, 1)

The target is 100% for administrative accounts and as close to 100% as operationally possible for all accounts. In a typical pre-course environment, this number is below 5% — almost nobody uses phishing-resistant methods unless they've been explicitly deployed. The gap between your current percentage and 100% is your exposure to AiTM — every sign-in using push notification, TOTP, or SMS is a sign-in that an AiTM proxy can intercept.

Track this monthly. After EI2 (authentication method deployment) and EI4 (authentication strength in Conditional Access), this number should show measurable improvement with each reporting cycle. The trajectory matters as much as the absolute number — a tenant moving from 3% to 40% in 90 days demonstrates program progress even though the target hasn't been reached.

Metric 2: Conditional Access evaluation rate

This is the percentage of successful sign-ins that were evaluated by at least one Conditional Access policy. Sign-ins without CA evaluation bypass your entire Zero Trust enforcement layer — they represent implicit trust that should have been explicitly verified.

// Metric 2: Conditional Access evaluation rate
SigninLogs
| where TimeGenerated > ago(30d)
| where ResultType == "0"
| extend CAPoliciesApplied = array_length(ConditionalAccessPolicies)
| summarize
    TotalSuccessful = count(),
    WithCA = countif(CAPoliciesApplied > 0),
    WithoutCA = countif(CAPoliciesApplied == 0)
| extend EvaluationRate = round(100.0 * WithCA / TotalSuccessful, 1)

The target is 99% or higher. Common causes of gaps include applications excluded from CA scope (legacy line-of-business applications, the Azure Management portal, custom applications that were added to exclusion lists during troubleshooting and never removed), user groups not targeted by any policy (service accounts, guest users, contractor accounts, break-glass accounts that are excluded by design but represent a known and monitored exception), and authentication flows like device code that aren't explicitly blocked in CA policy conditions.

When this metric shows 73%, the 27% gap is exactly where an attacker will target — the path of least resistance through your Zero Trust enforcement. An attacker who discovers that a specific legacy application isn't covered by CA can authenticate to that application's API and potentially access data through it without triggering any of your verification requirements. After EI3, EI4, and EI8, this number should approach 100% for interactive sign-ins. Service principal coverage requires Workload Identities Premium licensing and is addressed in EI10.

To identify exactly which applications and user groups are creating gaps, extend the query to group by AppDisplayName and UserType where CAPoliciesApplied == 0. The resulting list is your CA gap report — each entry is a specific application or user category that bypasses your Zero Trust enforcement entirely.

Metric 3: Privileged access standing exposure

This measures how many accounts have permanent administrative role assignments versus eligible (PIM-managed) assignments. Standing privilege is the attack surface — an attacker who compromises a permanently assigned Global Administrator gets those privileges immediately, with no additional challenge, no time limit, and no audit trail beyond the initial sign-in.

The math is straightforward. If your organization has 6 permanent Global Administrators and each account has credentials that could be compromised, you have 6 permanent targets for privilege escalation. With PIM, convert 4 of those to eligible assignments and keep 2 as permanent break-glass accounts. The standing exposure drops from 6 to 2. The eligible accounts still have access when they need it — they activate the role with additional MFA, provide a justification, and receive the assignment for a defined time window (typically 1-8 hours). If any of the 4 eligible accounts are compromised while the role is not active, the attacker gets a standard user account.

// Metric 3: Recent admin role assignments (90-day window)
// Each permanent assignment is standing privilege exposure
AuditLogs
| where TimeGenerated > ago(90d)
| where OperationName == "Add member to role"
| extend RoleName = tostring(TargetResources[0].modifiedProperties[1].newValue)
| where RoleName has_any ("Global Administrator", "Security Administrator",
    "Exchange Administrator", "SharePoint Administrator",
    "Privileged Role Administrator")
| extend AssignedUser = tostring(TargetResources[0].userPrincipalName)
| extend AssignedBy = tostring(InitiatedBy.user.userPrincipalName)
| project TimeGenerated, RoleName, AssignedUser, AssignedBy
// Review: each row is a role assignment in the last 90 days
// For current state, use Graph: Get-MgDirectoryRoleMember

This query shows role assignments made in the last 90 days — useful for identifying recent changes and assignments that may have been made during incidents and never revoked. The Midnight Blizzard breach exploited exactly this pattern: emergency access assigned during an incident, never cleaned up, eventually discovered by an attacker. For a complete picture of current standing assignments, the production approach uses the Graph API: Get-MgDirectoryRoleMember for each administrative role, filtered to distinguish permanent assignments from PIM-eligible assignments. EI6 teaches the full privileged access audit.

The target is 2 permanent Global Administrator accounts (break-glass accounts, monitored with Severity 1 alerts on any sign-in) and zero permanent assignments for all other administrative roles. Every other admin role should be PIM-eligible — activated on demand with justification, time-limited, and logged. Track this monthly. Any increase in permanent assignments should trigger an immediate review: was this an emergency that needs a follow-up governance action, or was this an unauthorized change that needs investigation?

Metric 4: Application permission sprawl

This measures the number of applications with high-privilege permissions — Mail.ReadWrite, Files.ReadWrite.All, Directory.ReadWrite.All, full_access_as_app, or equivalent — and specifically how many of those applications lack a defined owner. Applications without owners are ungoverned — nobody reviews their permissions, rotates their credentials, or monitors their activity.

The target is zero high-privilege applications without an accountable owner and a documented justification for the permissions they hold. The Midnight Blizzard breach succeeded because a dormant, over-privileged application existed without lifecycle management. Consent phishing succeeds because users can grant permissions without administrator review. Both attacks exploit permission sprawl.

# Metric 4: Applications with high-privilege permissions and no owner
$apps = Get-MgApplication -All
$highPrivApps = foreach ($app in $apps) {
    $perms = $app.RequiredResourceAccess.ResourceAccess.Id
    $owners = Get-MgApplicationOwner -ApplicationId $app.Id
    if ($perms -and $owners.Count -eq 0) {
        [PSCustomObject]@{
            AppName = $app.DisplayName
            PermissionCount = $perms.Count
            Owners = 0
        }
    }
}
$highPrivApps | Sort-Object PermissionCount -Descending
# Each ownerless app with permissions is an ungoverned attack surface

This PowerShell snippet identifies application registrations that have API permissions but no defined owner. The full permission audit in EI9 goes deeper — matching specific Graph API permission GUIDs to high-privilege operations like Mail.ReadWrite.All (GUID e2a3a72e-...) and Directory.ReadWrite.All (GUID 19dbc75e-...). The metric you track is the count of ownerless, high-privilege applications, trending toward zero over time.

Track this quarterly at minimum. Unlike the sign-in metrics that measure real-time posture, application permission sprawl changes slowly — applications are created, permissions are granted, and they persist indefinitely unless actively managed. A quarterly audit that identifies new ownerless applications and new high-privilege grants is the governance rhythm that keeps this metric under control.

Microsoft Secure Score for Identity

In addition to these four operational metrics, Microsoft provides Secure Score — a percentage-based posture assessment available in the Microsoft 365 Defender portal. Secure Score evaluates your tenant configuration against Microsoft's recommended security settings and provides a normalized score with specific improvement actions ranked by impact.

Secure Score is useful as a starting point and a conversation tool for leadership. It identifies common configuration gaps — accounts without MFA registered, legacy authentication not blocked, PIM not enabled, admin accounts without phishing-resistant authentication — and provides step-by-step remediation guidance. For organizations starting their identity security journey, working through the Secure Score recommendations is a structured way to address the most impactful configuration gaps first.

However, Secure Score measures configuration state, not operational effectiveness. A Conditional Access policy that exists but has scope gaps gets credit in Secure Score but doesn't provide the protection the score implies. A PIM configuration that has eligible roles but nobody uses the activation workflow gets credit even though standing access hasn't actually been reduced. An MFA registration campaign that registered 100% of users but deployed push notification instead of phishing-resistant methods gets full credit even though AiTM exposure hasn't changed.

The four metrics above measure what Secure Score doesn't: actual coverage rates based on live sign-in data, actual enforcement rates across real authentication events, and actual exposure levels as they exist in production. Secure Score tells you what you've configured. The KQL metrics tell you whether the configuration is working as intended. Use both — Secure Score for the configuration baseline and the KQL metrics for operational verification.

Entra ID Recommendations, available in the Entra admin center under Identity → Overview → Recommendations, provide more targeted guidance specific to your tenant. These recommendations are context-aware — they analyze your actual directory objects, policy configurations, and authentication patterns to surface issues like "3 applications have unused permissions that should be removed" or "5 users assigned Global Administrator haven't signed in for 60 days." Because they reference specific objects in your tenant, they're more immediately actionable than Secure Score's generic guidance. Check Entra Recommendations monthly alongside your KQL metrics.

Running your baseline

If you have access to a Sentinel workspace or Advanced Hunting in Microsoft 365 Defender, run these four metrics now. Record the numbers in a document you'll update monthly. These are your baseline — the starting point from which every module in this course produces measurable improvement.

A typical pre-course baseline looks like this. Metric 1 (phishing-resistant coverage) is below 5% — most organizations haven't deployed FIDO2 or passkeys beyond a small pilot group, if at all. Metric 2 (CA evaluation rate) is 60-80% — Conditional Access exists but has gaps in application coverage, user group targeting, and legacy authentication handling. Metric 3 (standing privilege) shows 4-8 permanent Global Administrators instead of the target 2 — accounts accumulated from incident responses, onboarding shortcuts, and "temporary" assignments that became permanent. Metric 4 (application permission sprawl) reveals 20-50 applications with high-privilege permissions, many without assigned owners or any record of why those permissions were granted.

These numbers are not unusual. They reflect the state of most M365 environments before systematic identity security work begins. The value of the baseline is not in judging the numbers — it's in having a measured starting point. When you complete EI2 and Metric 1 moves from 3% to 45%, that's demonstrable progress you can report to leadership. When EI6 moves Metric 3 from 6 permanent GAs to 2, that's a quantifiable reduction in attack surface.

If you don't have a lab environment yet, Section 0.10 walks you through the setup. The queries will be waiting when your environment is ready. The metrics become meaningful after EI1, when you learn to read sign-in logs fluently, and become actionable starting in EI2, when you deploy your first authentication method changes. By EI17, all four metrics should be at or near their targets.