E.1 Interpret the event chain
A proposed action, authorized request, issued call, accepted request, committed effect, reconciled effect, and verified business outcome are different states. Some resources commit asynchronously. The event schema should preserve those distinctions and permit an unresolved state rather than inventing completion.
| State | Meaning | Evidence |
|---|---|---|
| Proposed | An action is available for policy or review. | Normalized action and configuration. |
| Authorized or denied | Current authority and policy have been evaluated. | Grant, policy version, decision, reason, and approval if needed. |
| Issued | A resource request has been sent. | Call ID, idempotency key, payload digest, and timestamp. |
| Accepted | The resource has acknowledged processing. | Receipt and resource-specific acceptance semantics. |
| Committed / partial / uncertain | The effect is complete, incomplete, or not yet known. | External state and pending work. |
| Reconciled | Recorded and actual effect states agree or a discrepancy is explicitly resolved. | Independent match and correction record. |
| Verified outcome | The declared business postcondition has been independently checked. | Postcondition result, oracle, limitations, and task status. |
Use stable joins for workflow, agent and parent, initiating principal, configuration, grant, policy, approval, tool call, and receipt. Protect resolvable personal identifiers and full content separately from routine operational views. Collect only what the control purpose and applicable retention duty justify.
E.2 Metric definitions
Define the reporting window and strata before reporting. Preserve numerator, denominator, exclusions, unresolved cases, source grain, and uncertainty. The proposed dictionary below avoids collapsing utility, critical violations, reviewer capacity, and rights into one score.
M01. Unauthorized consequential commits
Unit and denominator: Count by severity; report eligible runs and consequential actions separately.
Definition: Count independently confirmed effects that violated current authority or required approval.
Evidence: Resource receipts joined to policy and approval.
Owner and interpretation: Platform and process owner. Any critical violation initiates containment.
M02. Safe task completion
Unit and denominator: Eligible runs; report excluded and unresolved runs.
Definition: Authorized successful runs without a defined prohibited effect / eligible runs.
Evidence: Same-run rubric; independent postcondition, oracle and limitations (E.1).
Owner and interpretation: Business owner. Use precommitted workflow threshold and confidence method.
M03. Timely prevention
Unit and denominator: Labeled harmful cases under the specified scenario design.
Definition: Harmful cases prevented before effect within the precommitted criticality-specific deadline / labeled harmful cases.
Evidence: Labels, effective intervention time, resource state.
Owner and interpretation: Security operations. Report misses and late interventions by criticality.
M04. Unnecessary interruption
Unit and denominator: Labeled benign cases.
Definition: Benign cases erroneously interrupted / benign cases.
Evidence: Adjudicated benign labels and actual hold/deny events.
Owner and interpretation: Business operations. Define acceptable burden against staffing and service commitments.
M05. Review queue delay
Unit and denominator: Cases requiring review, by stage and severity.
Definition: Review-start minus queue-entry; review-completion minus queue-entry. Report median, p95, tails, and overdue count.
Evidence: Queue, reviewer, and completion timestamps.
Owner and interpretation: Review operations. Define action-class deadlines and peak-load stop conditions.
M06. Evidence reconciliation
Unit and denominator: Consequential actions, including uncertain and partial ones.
Definition: Actions with the required matched authority, approval, decision, effect, and outcome evidence / consequential actions.
Evidence: Independent action inventory and event matches.
Owner and interpretation: Platform and assurance. Unreconstructable critical effects reopen assurance.
M07. Scope or revocation failures
Unit and denominator: Negative tests and production actions reported as separate populations.
Definition: Count prohibited acceptances; do not blend adversarial samples with production prevalence.
Evidence: Grant state, decisions, and resource effects.
Owner and interpretation: Identity and resource owners. A material acceptance initiates authority reduction and repair.
M08. Configuration mismatch
Unit and denominator: Inspected runs with a declared approved configuration.
Definition: Runs whose material deployed configuration differs from approved configuration / inspected runs.
Evidence: Approved manifest and production identity.
Owner and interpretation: Platform and validation. Separate unapproved mismatch from opaque upstream version uncertainty.
M09. Recovery performance
Unit and denominator: Each incident or drill; no averaging across unlike effects without scope.
Definition: Time to effective halt; active descendants at verification; pending/partial effects reconciled.
Evidence: Resource queries, queue state, and delegation tree.
Owner and interpretation: Incident response. Compare with approved recovery and containment objectives.
M10. Customer stage outcomes
Unit and denominator: Eligible entrants for each defined stage and relevant cohort.
Definition: Report entry, burden, exclusion, errors, delay, correction, and remedy with explicit counts and uncertainty.
Evidence: Stage lineage, lawful cohort labels, adjudication.
Owner and interpretation: Business and compliance. Investigate material unexplained differences and rights failures.
E.3 Threshold and sampling cautions
Do not copy a benchmark threshold into production without the workflow population, task, and consequence justification. Precommit operating decisions, then report the observed evidence. A monitor threshold selects a tradeoff between misses and false interruption. A queue percentile can conceal a critical overdue case. A high reconciliation percentage can conceal the one unreconstructable effect that requires containment.
The exact zero-event example in chapter 9 requires independent identically distributed binary observations. It cannot resolve missing hazard coverage or transform a hand-selected adversarial set into a production prevalence estimate.