# Product Quality Flow Coverage Operating Model v1

Status: draft
Owner: Product Quality / Architecture Control
Primary Fairway task: `UX-UAT-FLOW-COVERAGE-GAP-ANALYSIS-001`

## Purpose

The Product Quality / Flow Coverage track exists to prevent broad UAT and user
testing from becoming the first place where missing product behavior, missing
UX, missing APIs, missing fixtures, or missing cleanup paths are discovered.

The track turns product intent into a durable coverage map. A UAT pass is only
meaningful when the relevant persona flows, prerequisites, recovery paths,
automation, and evidence owners are already mapped.

This model also applies to operator workflows and approval-gated drills. The
MFA live-drill loop showed that strong architecture/design documents are not
enough when provider API shape, browser/runtime surface, rollback ownership,
review handoff, and evidence dependencies are discovered only during the live
window. Dependency discovery must be an upstream deliverable.

## Operating Principle

UAT should validate integrated behavior. It should not be the first discovery
loop for:

- absent UX behavior;
- missing API or CLI contract support;
- missing fixtures or seed data;
- hidden environment prerequisites;
- untested empty states or blocked states;
- cleanup, rollback, or decommission gaps;
- user-safe error and recovery gaps;
- harness defects that hide failures.

The same rule applies to live drills, deploys, and production-readiness gates:
they should validate an already-mapped dependency graph. They should not be the
first place where the team learns that a provider Admin API endpoint shape,
exact browser execution surface, CI runner capability, DNS/edge route, identity
fixture, rollback target, or Fairway handback path is missing.

When broad UAT finds one of these issues, record it as an escape from the
expected earlier gate and create a scoped Fairway task using the standard
finding prefixes.

## Required Flow Coverage Shape

Every critical product flow must be represented as a row in the flow coverage
matrix before it is declared ready for broad UAT.

Required fields:

| Field | Meaning |
|---|---|
| `flow_id` | Stable flow ID, for example `FLOW-COMPUTE-LAUNCH-001`. |
| `persona` | User, admin, app developer, ops, security, finance, or governance persona. |
| `entry_point` | Natural product, CLI, or API entry point. |
| `happy_path` | Expected successful user or operator path. |
| `empty_state` | What happens when the user lacks required data, capacity, credentials, or artifacts. |
| `blocked_state` | How the product explains unavailable prerequisites or policy blocks. |
| `in_place_recovery` | How the user/operator fixes the blocker without losing context. |
| `negative_path` | Expected user-safe error behavior and correlation/evidence capture. |
| `cleanup_path` | Release, decommission, rollback, delete, or archive path. |
| `ux_anchor` | UX journey/spec section, or explicit design gap. |
| `contract_anchor` | OpenAPI, AsyncAPI, CLI command, SDK contract, or explicit contract gap. |
| `automation_anchor` | Unit, integration, e2e, smoke, or UAT coverage. |
| `fixture_prerequisite` | Required seed data, external resource, provider capacity, credentials, or none. |
| `dependency_map` | Required identities, permissions, fixtures, provider/API surfaces, DNS/edge routes, browser/runtime surfaces, CI/CD or deploy lanes, rollback/cleanup owners, and evidence owners. |
| `preflight_proof` | Non-live or low-risk command/artifact that proves dependencies before broad UAT, deploy, or a live drill. |
| `environment_lane` | `kind`, `dev`, `demo`, or future lane with reason. |
| `status` | `ready_for_uat`, `needs_design`, `needs_contract`, `needs_runtime_fix`, `needs_fixture`, `needs_env_fix`, `needs_e2e`, `accepted_gap`, or `blocked`. |
| `gap_task` | Owning Fairway task when status is not `ready_for_uat`. |
| `detected_by` / `expected_gate` | Classifiers from `DevSecOps_Escape_Rate_Classifier_v1.md`. |

## Dependency Mapping Gate

Before implementation, broad UAT, deploy-readiness validation, or an
approval-gated live drill begins for a critical flow, create or update a
dependency map. The map can live in this matrix, a decision packet, or a
task-specific readiness bundle, but it must be linked from the Fairway task.

Minimum fields:

| Dependency | Required answer |
|---|---|
| Flow/subpaths | Happy, empty, blocked, in-place recovery, negative/error, cleanup, rollback, and evidence capture. |
| Product/contract owners | UX, OpenAPI/AsyncAPI, CLI/SDK, backend service, worker, provider adapter, frontend shell, or accepted gap. |
| Identity and permissions | Personas, service accounts, roles/groups, break-glass accounts, reviewer/approver roles, and missing access. |
| Fixtures and state | Seed data, retained fixtures, disposable resources, realm/client/user/group setup, capacity, and cleanup manifest. |
| Provider/runtime surfaces | Keycloak/Cloudflare/Pomerium/MAAS/Proxmox/Kubernetes/browser/CI runner surfaces and exact command or endpoint shape. |
| Environment routes | DNS, edge/proxy, auth callback, TLS/CA, public/private route, and lane-specific overrides. |
| Preflight proof | Non-live classifier, dry-run, readback, smoke, launch probe, or environment check that proves the dependency before broad validation. |
| Stop/rollback owners | Named owner for rollback, cleanup, evidence, and escalation or explicit accepted exception. |
| Residual risk | What remains unproven, why it is acceptable, expiry, and owning follow-up task. |

If the map cannot answer one of these fields, the flow is not ready for broad
validation. Create a scoped follow-up task and stop at the earliest safe gate.

## UAT Summary Evidence Gate

Full UAT summary evidence must preserve the Product Quality flow layer, not
only row-oriented `UAT-*` matrix IDs. Each structured UAT result row that is
part of a P0/P1 flow must include:

| Field | Required answer |
|---|---|
| `matrix_id` | The mapped UAT row, for example `UAT-COMPUTE-001`. |
| `flow_id` | The Product Quality flow ID, for example `FLOW-COMPUTE-LAUNCH-001`. |
| `flow_subpaths.happy` | Happy-path result and evidence. |
| `flow_subpaths.empty` | Empty-state result and evidence. |
| `flow_subpaths.blocked` | Blocked-prerequisite result and evidence. |
| `flow_subpaths.recovery` | In-place recovery result and evidence. |
| `flow_subpaths.negative` | Negative/error-path result and evidence. |
| `flow_subpaths.cleanup` | Cleanup, rollback, or retained-fixture closeout result and evidence. |
| `flow_subpaths.fixture` | Fixture prerequisite proof or approved skip. |
| `flow_subpaths.environment` | Environment-lane prerequisite proof or approved skip. |

Missing subpaths must appear in the deterministic UAT summary as blocked
coverage rows. Skipped subpaths are valid only when they include an owner,
Fairway follow-up task, expiry, and residual-risk reason. A full UAT run cannot
be treated as green when a required flow subpath is absent, blocked without
owner/action, or skipped without expiry.

Summary-only or otherwise unstructured evidence that lacks Product Quality
`flow_id` and subpath coverage is non-authoritative for P0/P1 flow readiness.
The deterministic summary must either block readiness or explicitly label the
packet as non-readiness evidence so operators cannot treat a green table of
checks as broad UAT completion.

## Flow Contract Review Gate

MFA drills and similar approval-gated browser journeys require an additional
flow-contract review before exact-window approval. Local reviews can prove that
a delta is safe without proving that the whole journey should reach the
expected browser state. Product Quality / Integration must review the causal
model from provider configuration to browser outcome.

Minimum flow-contract inputs:

| Input | Required answer |
|---|---|
| Provider state | Realm/profile, client, redirect/callback route, browser flow tree, requirement levels, and provider version. |
| Identity graph | Users, groups, roles, required actions, service/non-human exclusions, and break-glass path. |
| Expected browser states | Ordered state sequence per persona, including login form, MFA challenge/enrollment, callback, blocked, excluded, and error states. |
| Helper state machine | Classifiers, screenshots/log artifacts, unknown-state handling, and redaction boundaries. |
| Non-live proof | Disposable or isolated-environment proof that the expected sequence is reachable before live mutation. |
| Stop/rollback proof | Stop conditions, rollback/delete owner, source-realm readback, cleanup, and evidence handback. |

Product Quality / Integration approval must answer whether the supplied
configuration should cause the expected end-to-end journey. If the packet only
shows that the latest helper or script delta is safe, but does not explain why
the full browser journey should reach MFA challenge, enrollment, callback, or a
known blocked state, request changes.

The canonical MFA template is
`doc/operations/MFA_Flow_Contract_Product_Quality_Review_Model_v1.md`.

## Implementation Entry Rule

Architecture and design documents describe the target. They do not by
themselves prove the execution path is ready. For new or changed critical
flows, implementation may start only when one of these is true:

- the dependency map is complete enough for `ready_for_dev`;
- the work is explicitly scoped to produce the missing dependency map or
  preflight proof;
- Architecture Control records an `accepted_gap` with owner, reason, expiry,
  and Fairway task.

Broad UAT, user testing, deploy, or live drill scheduling requires the stronger
`ready_for_uat`/readiness-bundle posture, not merely `ready_for_dev`.

## Coverage Levels

Use these levels to describe the strongest evidence available for each flow:

| Level | Evidence |
|---|---|
| L0 | Product intent and UX journey are documented. |
| L1 | API, CLI, SDK, or event contract exists and matches the UX. |
| L2 | Unit or integration coverage proves the service behavior. |
| L3 | Frontend e2e or non-live smoke proves the user/operator path. |
| L4 | Environment-specific UAT proves the integrated path on `kind` or `dev`. |
| L5 | Cleanup, audit, observability, and rollback evidence are captured. |

Critical flows should not start broad UAT below L3 unless the gap is explicitly
accepted with owner, reason, expiry, and Fairway follow-up.

## Priority Rules

- `P0`: flow blocks login, account/security posture, compute launch/release,
  terminal/connect, billing safety, environment cleanup, privileged ops, or
  launch-readiness evidence.
- `P1`: flow blocks a supported product family, app runtime, scheduler,
  storage, project/IAM administration, observability, or developer contract.
- `P2`: flow improves usability, diagnostics, docs, or secondary coverage but
  has an accepted workaround.

P0 and P1 gaps must be visible before broad UAT starts. They do not all need to
be fixed before every UAT attempt, but they must be named, owned, and classified
so a green UAT summary cannot hide them.

## Fairway Task Routing

Use the standard finding prefixes:

| Prefix | Use For |
|---|---|
| `UAT-BUG-*` | Product/runtime behavior found by UAT or flow review. |
| `HARNESS-FIX-*` | Test harness false positives, false negatives, flakes, or sequencing defects. |
| `CI-FIX-*` | Build, test, lint, generated contract, or CI runner failure. |
| `CD-FIX-*` | Promotion, deploy script, rollout, image freshness, or cluster apply failure. |
| `OPS-FIX-*` | Environment, capacity, credential, backup, observability, or network issue. |
| `DOC-FIX-*` | Runbook, UAT guide, or operator document mismatch. |
| Product/UX governance task | Missing UX journey, IA, product behavior, or acceptance model. |

Each actionable finding should include `detected_by` and `expected_gate`.

## Gate Usage

Before a broad UAT run, Product Quality should provide:

- the set of flow IDs in scope;
- current coverage level for each flow;
- P0/P1 gaps and owning Fairway tasks;
- fixtures and environment prerequisites;
- accepted skips with owner, reason, expiry, and evidence path.

After a broad UAT run, every failure should be classified against the coverage
matrix. If the failure was a missing flow, missing UX, missing contract, missing
fixture, or harness defect, update the Product Quality matrix and record the
escape so the same class of issue moves earlier in the process next time.