A skipped step is a successful step, so the run status cannot distinguish these two outcomes.
An analyst mentions in passing that incidents have stopped arriving with IP reputation attached. You open the playbook that does it. Four hundred and twelve runs in six weeks, every one succeeded, no failures, running right up to this afternoon. The obvious conclusion is that the analyst is misremembering.
They are not. The playbook has been reaching its second action and stopping there since the beginning of June, and the platform has reported that as success four hundred and twelve times.
Completion is not the same as action
A workflow engine reports whether the flow reached its end. It does not report whether the flow did anything, and those are different questions with the same answer in the status field.
Consider what happens when a condition evaluates false. The engine evaluates the condition, takes the branch with nothing in it, reaches the end of the definition, and records a successful run. Every step after the condition is marked skipped, and skipped is not failed. The run is green, the duration is short, the error count is zero, and the automation has done nothing.
This is not a bug in anybody's platform. It is the correct behaviour for a workflow engine and it is the reason SOAR monitoring built on run-success rate tells you almost nothing. A playbook that has never run and a playbook that runs hourly and stops at step two are both reported as healthy, because both are.
The query that finds it
Actions completed per run, per playbook, sorted ascending. In Sentinel and Logic Apps that means reading the diagnostic logs rather than the portal.
// Actions per run, per playbook. The number to compare against is the
// count of actions in the playbook definition: a six-action playbook
// averaging 2.0 is stopping a third of the way through, every time.
// Sort ascending. The interesting row is the bottom one.
AzureDiagnostics
| where TimeGenerated > ago(30d)
| where ResourceProvider == "MICROSOFT.LOGIC"
| where Category == "WorkflowRuntime"
| summarize Runs = dcountif(resource_runId_s,
OperationName endswith "workflowRunStarted"),
ActionsSucceeded = countif(OperationName endswith "workflowActionCompleted"
and status_s == "Succeeded"),
ActionsFailed = countif(OperationName endswith "workflowActionCompleted"
and status_s != "Succeeded")
by Playbook = resource_workflowName_s
| extend ActionsPerRun = round(1.0 * ActionsSucceeded / Runs, 1)
| where Runs > 5
| order by ActionsPerRun ascThe threshold is the part people get wrong. There is no universal floor, because two actions per run is perfect for a two-action playbook and catastrophic for a six-action one. The comparison has to be against each playbook's own step count, or against its own history, and a fixed number like "fewer than three actions per run" flags every small playbook in the estate and trains everyone to ignore the alert.
The version that works in practice compares a playbook against itself over time:
// Behaviour change rather than an absolute floor. A playbook whose
// actions-per-run halves between two periods has changed what it does,
// and nothing in the platform notices.
let baseline = AzureDiagnostics
| where TimeGenerated between (ago(60d) .. ago(30d))
| where ResourceProvider == "MICROSOFT.LOGIC"
| summarize Runs = dcountif(resource_runId_s, OperationName endswith "workflowRunStarted"),
Actions = countif(OperationName endswith "workflowActionCompleted" and status_s == "Succeeded")
by Playbook = resource_workflowName_s
| extend BaselineRatio = 1.0 * Actions / Runs
| project Playbook, BaselineRatio;
AzureDiagnostics
| where TimeGenerated > ago(30d)
| where ResourceProvider == "MICROSOFT.LOGIC"
| summarize Runs = dcountif(resource_runId_s, OperationName endswith "workflowRunStarted"),
Actions = countif(OperationName endswith "workflowActionCompleted" and status_s == "Succeeded")
by Playbook = resource_workflowName_s
| extend CurrentRatio = 1.0 * Actions / Runs
| join kind=inner baseline on Playbook
| extend DropPct = round(100.0 * (BaselineRatio - CurrentRatio) / BaselineRatio, 1)
| where DropPct > 25
| project Playbook, BaselineRatio = round(BaselineRatio, 1),
CurrentRatio = round(CurrentRatio, 1), DropPct
| order by DropPct descIf your diagnostic logs land in Splunk rather than a Log Analytics workspace, the same shape holds against the ingested events:
index=azure sourcetype="azure:diagnostics" ResourceProvider="MICROSOFT.LOGIC"
| eval is_start=if(match(OperationName, "workflowRunStarted$"), 1, 0)
| eval is_action=if(match(OperationName, "workflowActionCompleted$") AND status_s="Succeeded", 1, 0)
| stats dc(eval(if(is_start=1, resource_runId_s, null()))) as runs,
sum(is_action) as actions by resource_workflowName_s
| eval actions_per_run=round(actions/runs, 1)
| where runs > 5
| sort + actions_per_runFour causes, four different fixes
A low ratio tells you the flow stops early. It does not tell you why, and the four causes need separating because they belong to different people.
A condition reading a field that changed shape. The most common by a wide margin, and it rarely announces itself. An entity mapping gets simplified, a licence change stops populating a column, a connector update renames a property, and a condition that tested for something starts evaluating false on every run. The playbook was never edited, which is why nobody looks at it. Check whether the field the condition reads is still populated before checking anything else.
The diagnostic for that first cause is a field population check, and it is more useful than reading the condition. If a column was populated last month and is empty this month, you have both the cause and the date, and the date is what you correlate against whatever changed.
// Field population by week. A value going from 100 percent to zero
// between two consecutive weeks is a configuration change somebody
// made on a particular day, not gradual degradation.
// Adapt the countif list to the fields your playbook conditions read.
SecurityIncident
| where TimeGenerated > ago(120d)
| summarize Incidents = count(),
WithOwner = countif(isnotempty(tostring(Owner.userPrincipalName))),
WithSeverity = countif(isnotempty(Severity)),
WithLabels = countif(tostring(Labels) != "[]"),
WithAlerts = countif(array_length(todynamic(AlertIds)) > 0)
by Week = bin(TimeGenerated, 7d)
| extend OwnerPct = round(100.0 * WithOwner / Incidents, 1),
LabelsPct = round(100.0 * WithLabels / Incidents, 1),
AlertsPct = round(100.0 * WithAlerts / Incidents, 1)
| project Week, Incidents, OwnerPct, LabelsPct, AlertsPct
| order by Week ascA clean transition rather than a decline is what identifies the cause. Gradual movement looks like a connector sampling change or a licence taking effect across a population. A value dropping from 100 to zero between two weeks narrows the search from an open question to a diff, and it is why the next step is looking for a modification date rather than looking for a fault. Nothing is broken in the sense of malfunctioning, which is exactly why nothing alerted.
An earlier automation rule closing the incident. Rules run in order, and a rule that closes or resolves an incident prevents every later rule from seeing it. The affected playbook does not run badly, it does not run at all, and its run history simply stops. This is invisible from the playbook and obvious from the rule list.
# Rule order and what each rule does. A closing action at a low order
# number is a complete explanation for every playbook below it, and this
# is the check that takes ten seconds and gets skipped.
$sub = (Get-AzContext).Subscription.Id
$base = "https://management.azure.com/subscriptions/$sub/resourceGroups/rg-soc" +
"/providers/Microsoft.OperationalInsights/workspaces/law-soc" +
"/providers/Microsoft.SecurityInsights/automationRules"
(Invoke-AzRestMethod -Uri "$base`?api-version=2025-09-01" -Method GET).Content |
ConvertFrom-Json | Select-Object -ExpandProperty value |
Select-Object @{n='Order';e={$_.properties.order}},
@{n='Rule';e={$_.properties.displayName}},
@{n='Enabled';e={$_.properties.triggeringLogic.isEnabled}},
@{n='Closes';e={[bool]($_.properties.actions.actionConfiguration.status -eq 'Closed')}} |
Sort-Object OrderAn exception caught and logged below the alerting threshold. A catch block that logs and continues is correct behaviour for an enrichment, where one failed lookup should not lose the other three. Copied into a collection or a remediation, the same three lines produce a run that reports success having skipped work. The signal is the handled-exception count rising while the failure count stays at zero, and warning level puts it below everything that reports.
A connection that lost its permission. A revoked consent or an expired secret produces an error that many connectors hand back to the flow rather than throwing, and a flow with error handling then swallows it. The run completes. The action did not.
The same failure in bulk, where it costs more
Everything above concerns a playbook that does nothing. The variant worth knowing about is a playbook or function that does some of what it should, because the reporting unit and the damage unit are different.
A loop over a list produces one invocation regardless of whether the list held two items or two hundred. If the loop catches an exception per iteration and continues, which is the pattern most people write, it reaches the end and reports success having skipped whatever failed. Duration looks normal. The error count is zero. And the only record that anything was missed is a warning nobody alerts on.
The consequence scales the wrong way. A bulk operation across three entities usually succeeds completely, because three requests do not meet a rate limit. The same operation across twenty meets a per-tenant throttle partway through, so the incidents most likely to lose work are the largest ones, which are also the ones where the work mattered most.
The check is a set difference rather than a ratio: what the automation was given against what it actually did.
// Intended against actual, for anything that acts on a list. The audit
// is the only place the unit is the entity rather than the run, and a
// short count means the remainder was never acted on.
let runStart = datetime("2026-07-24T14:31:00Z");
let intended = 19; // from the trigger query or the playbook input
AuditLogs
| where TimeGenerated between (runStart .. (runStart + 15m))
| extend Actor = tostring(InitiatedBy.app.displayName),
Target = tostring(TargetResources[0].userPrincipalName)
| where isnotempty(Actor)
| summarize Acted = dcount(Target), Actions = count() by Actor
| extend Intended = intended, Missed = intended - ActedIf that returns seven where the input was nineteen, twelve entities were never touched and the run reported success. Retry with backoff is the fix, and logging the item count is what makes the gap visible next time.
What healthy looks like
The numbers below are worth writing down for your own estate rather than borrowing, but they give you a starting shape.
| Measure | Healthy | Investigate |
|---|---|---|
| Actions per run against the playbook's step count | Within one of the expected count | More than 30 percent below |
| Ratio change against the previous month | Under 10 percent movement | A drop over 25 percent |
| Handled exceptions per 100 runs | Near zero for tier 1 enrichment | Any sustained rate on a containment playbook |
| Runs with zero actions | Zero, unless the trigger is deliberately broad | Any, on a playbook that takes an action |
| Playbooks with no run in 30 days | Zero, or a documented reason each | Any undocumented |
The fourth row is the one that catches the case in the opening paragraph. A playbook that completes with zero actions is not a slow playbook or a badly tuned playbook. It is a playbook that is not running your automation, and it will report success indefinitely.
What to do this week
Run the actions-per-run query and sort ascending. Ten minutes. You are looking at the bottom five rows, not the average.
Open each of those playbooks and count its actions. The ratio means nothing without the denominator, and the denominator is not in any log. A playbook with two actions averaging 2.0 is fine. A playbook with eight averaging 2.0 has been broken for however long the trend holds.
For anything materially below its step count, expand one run and read the branch that was not taken. The portal shows skipped steps as successful, so read the condition rather than the status. Then check whether the field that condition reads is still populated in recent incidents.
Check your automation rule order for anything that closes incidents. If a closing rule sits above a playbook-running rule, that playbook has not run since the closing rule was added, and its run history will simply stop rather than show failures.
Replace run-success rate with the ratio in your automation review. Success rate is the metric that reported four hundred and twelve healthy runs on a playbook that had done nothing for six weeks. Actions per run would have surfaced it in the first week.
Take this further
Microsoft's Logic Apps monitoring documentation covers enabling the diagnostic logs these queries read, which are off by default on many workspaces. The automation rules ordering behaviour is documented but easy to miss, and it is the cause people find last. For the exception-handling half, the Logic Apps exception handling patterns article explains scopes and what a caught failure does to the run status.
The broader point applies past SOAR. Any system where the reported unit and the useful unit differ will do this: a loop reports one invocation regardless of whether it processed two items or two hundred, a bulk operation reports success having skipped a third of its list, and a scheduled query reports completion whether or not it returned rows. When you inherit a monitoring dashboard, the first question worth asking is what the green light is actually measuring.