# Product Quality Flow Coverage Matrix v1

Status: draft gate packet for `UX-UAT-FLOW-COVERAGE-GAP-ANALYSIS-001`
Owner: Product Quality / Architecture Control

## Purpose

This matrix maps P0/P1 product flows to UX intent, contracts, automation, UAT
rows, fixtures, environment prerequisites, and current gaps. It is the
product-flow layer that sits above the row-oriented UAT completion matrix.

Broad UAT should not start for a critical flow unless this matrix shows at
least L3 coverage or an explicit owner-backed exception.

Coverage levels:

- L0: product intent and UX journey documented.
- L1: API, CLI, SDK, or event contract exists and matches the UX.
- L2: unit or integration coverage proves service behavior.
- L3: frontend e2e or non-live smoke proves the user/operator path.
- L4: environment-specific UAT proves the integrated path on `kind` or `dev`.
- L5: cleanup, audit, observability, and rollback evidence are captured.

Statuses:

- `ready_for_broad_uat`: adequate for broad UAT in the named lane.
- `blocked`: must not be claimed ready until the owning task closes or an
  approved exception is recorded.
- `accepted_gap`: known gap with owner, reason, expiry, and follow-up.
- `needs_coverage`: behavior exists but required automation/UAT evidence is
  missing.
- `needs_design`: UX journey/spec is not sufficient.
- `needs_contract`: API/CLI/SDK contract support is absent or ambiguous.
- `needs_fixture`: required seed/disposable state is missing.
- `needs_env_fix`: environment prerequisite is missing or stale.

## P0 Flow Matrix

| Flow ID | Persona | Entry Point | Happy Path | Empty State | Blocked State | Recovery | Negative Path | Cleanup | UX Anchor | Contract / CLI Anchor | Automation Anchor | UAT Row | Fixture / Env | Lane | Level | Status | Owner | Gap Task |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FLOW-AUTH-SESSION-001 | End user / admin | `/auth/login`, protected route redirect | Work SSO or personal login establishes session, refresh keeps access, logout clears session | No session redirects to login with return target | Provider/auth dependency failure renders product-owned error with correlation ID | Retry sign-in; tenant/admin setup guidance for federation failures | Failed exchange hides raw provider text and preserves correlation ID | Logout clears local session | `UX_Journeys.md` Authentication; `UX_Implementation_Spec.md` Auth slice | OIDC authorize/exchange, personal login/signup, refresh, logout | `auth-login.spec.ts`, `auth-guard.spec.ts`, auth callback/session unit tests | `UAT-AUTH-001` | Keycloak/OIDC profile, user/admin personas | kind/dev | L4 | ready_for_broad_uat | frontend/backend | none |
| FLOW-ACCOUNT-MFA-001 | End user / admin/security | `/account/security`, `/account/sessions`, `/platform/iam` | User sees product-owned MFA lifecycle state, can set up a first factor or add a backup, can submit support-assisted removal/recovery requests, and operators see read-only MFA readiness/evidence posture | Provider-pending/unavailable posture renders product-owned explanation without governance noise; provider readback distinguishes observed TOTP/WebAuthn from unqueried/unavailable when configured | Production packet must exclude local sensitive-operation step-up unless a future alternate claim/edge/step-up design is accepted | Provider setup action, existing-factor posture, support-assisted remove/recovery request submission, sessions cleanup, runbook/evidence posture, and production rollout packet | No raw auth/provider errors; no fake `amr`/`acr` claims; service accounts/API keys cannot satisfy human MFA; daily platform admins are not exempt from MFA | Logout/session cleanup only; no silent factor bypass; locked-out reset and support/admin queue fulfillment remain separate controlled flows | Account/security journey; MFA posture read-model decision; Platform IAM readiness decision; `MFA_User_Factor_Setup_Manage_Flow_Coverage_v1.md`; `MFA_Factor_Lifecycle_UAT_Coverage_v1.md` | `GET /api/v1/account/security`; `POST /api/v1/account/security/mfa/recovery-requests`; `GET /api/v1/platform/iam/mfa-readiness`; current OIDC `amr`/`acr` claim proof closed with `keep_deferral` | `v3-account-mfa.spec.ts`, `v3-storage-access-account.spec.ts`, `v3-live-mfa-product.spec.ts`, `v3-account-security-sessions.test.tsx`, `v3-admin-platform.spec.ts`, `v3-platform-pages.test.tsx`, `keycloak_mfa_nonlive_claim_contract_proof_runner.sh`, `keycloak_mfa_nonlive_claim_contract_collect_sanitized.sh` | `UAT-IAM-001`, `UAT-AUTH-001`, `UAT-OPS-001`, `UAT-MFA-USER-001`, `UAT-MFA-OPS-001`, `UAT-MFA-OPS-002`, `UAT-MFA-OPS-003` | kind/dev UI/UAT evidence now covers setup/add-backup entry, provider return states, signed-in support-assisted request submission, focused factor-present manage-flow e2e, user-facing branding cleanup, sensitive platform IAM step-up fail-closed behavior, superadmin non-production exception posture, and daily-admin versus break-glass policy separation; kind `api/web` deploy passed at `148ef67974582fa0314510f813c2d241853f4202`; full product-ready closure still waits on enrolled-factor live proof, queue fulfillment, locked-out recovery, provider unavailable proof, non-human exclusion, provider internals, and current-session assurance | kind/dev | L4 | accepted_gap | security/backend/frontend/ops/product-quality | `PRODUCT-GAP-IAM-MFA-PROVIDER-FACTOR-READBACK-001`; `PRODUCT-GAP-IAM-MFA-FACTOR-MANAGE-FLOW-001`; `PRODUCT-GAP-IAM-MFA-FACTOR-REMOVE-DISABLE-FLOW-001`; `PRODUCT-GAP-IAM-MFA-FACTOR-RECOVERY-FLOW-001`; `UAT-BUG-IAM-MFA-PROVIDER-RETURN-FLOW-001`; `IAM-MFA-FULL-FUNCTIONAL-UAT-001` |
| FLOW-PROJECT-IAM-001 | Tenant/project admin | `/access/projects`, `/access/service-accounts`, shell project context | Create/select/update/delete project; create/delete service account; authz/audit visible | Empty tenant has create path and default department hidden for small tenants | Role/tenant denial rendered as restricted state | Create project/service account inline from access surfaces; preserve context | 403/validation errors shown with product copy and correlation ID | Delete project/service account through explicit destructive path | Project lifecycle; department UX gate | Project APIs; service-account APIs; role/authz smoke | `v3-live-kind.spec.ts`, `v3-access-projects.test.tsx`, `v3-access-governance.test.tsx`, `role_authz_smoke.sh` | `UAT-IAM-001` | Disposable tenant/project/service account; role-bound personas | kind/dev | L5 | ready_for_broad_uat | frontend/backend/ops | none |
| FLOW-SSH-KEY-001 | End user / project admin | `/access/credentials`, `/account/security`, launch wizard | Add/promote/delete SSH key and use it for launch/connect | No key shows create path without losing launch draft | Bare-metal/compute launch blocks before mutation when no resolvable key | Inline key creation or route to credentials with return context | Invalid key validation error is structured | Delete disposable key after UAT | Operate active allocation; terminal/key UX | SSH key APIs; allocation create key validation | `v3-live-account-security-ssh.spec.ts`, `v3-compute-apps.spec.ts`, `launch-inline-dependencies.test.tsx` | `UAT-IAM-001`, `UAT-COMPUTE-001` | Disposable SSH key | kind/dev | L5 | ready_for_broad_uat | frontend/backend | `PSSM-ALLOCATION-CREATE-SSH-KEY-VALIDATION-001` closed |
| FLOW-COMPUTE-LAUNCH-001 | End user | `/compute`, `/launch/compute`, `/tasks/:id` | User selects SKU, satisfies readiness, submits once, reaches task/workload | No SKU/key/capacity shows recoverable readiness actions | Sold out/capacity/insufficient-balance blocks before mutation | Inline SSH key and readiness actions; rerun precheck | User-safe API errors; no row disappearance | Release allocation and cleanup manifest | Launch workload; async UX | `POST /api/v1/allocations`, v3 launch compute APIs, idempotency | `v3-compute-apps.spec.ts`, `v3-live-compute-launch.spec.ts`, `kind_uat_prereq_baseline.sh` | `UAT-COMPUTE-001` | Provider capacity, SSH key, project, balance | kind/dev | L4 | ready_for_broad_uat | backend/ops/frontend | `OPS-FIX-KIND-COMPUTE-CAPACITY-PREREQ-001` closed |
| FLOW-COMPUTE-RELEASE-001 | End user / ops | `/workloads/:id`, allocation compatibility route | Release moves to releasing/released with visible status transition | No active allocation still exposes history filters | `release_failed` shows reason, billing-stopped note, retry action | Retry release; admin force-release path | Release errors classified and correlated | Cleanup manifest proves released/provider cleanup | Operate allocation; release lifecycle UX | Allocation release; admin force-release | `v3-live-compute-launch.spec.ts`, workload tests, cleanup validator | `UAT-COMPUTE-002` | Disposable allocation; provider cleanup access | kind/dev | L5 | ready_for_broad_uat | backend/ops/frontend | `PSSM-UAT-POST-RUN-RESOURCE-CLEANUP-001` closed |
| FLOW-TERMINAL-CONNECT-001 | End user / ops | `/workloads/:id?tab=connect`, `/terminal/popout` | Mint single-use token, connect WS, receive `session_ready`, command output works | No active allocation disables terminal and points to launch/workloads | Node stream timeout/stale task backlog classified before broad UAT | Remint token/retry; node-agent task backlog recovery owner | No query-string tokens; terminal control frame errors are user-safe | Close WS; cleanup continues on failure | Terminal UX | Terminal token API; `/ws/terminal/{allocation_id}` | `v3-live-ws-session.spec.ts`, `TerminalPanel.test.tsx`, `terminal_remote_smoke.sh`, `ws_terminal_smoke.go` | `UAT-TERMINAL-001` | Existing/fresh active allocation; terminal route | kind/dev | L5 | ready_for_broad_uat | backend/ops/frontend | `OPS-FIX-KIND-NODE-AGENT-TASK-BACKLOG-TERMINAL-STREAM-001` closed |
| FLOW-BILLING-SAFETY-001 | End user / finance/admin | `/account/billing`, platform finance, launch readiness | Balance/usage visible, checkout/portal starts, low balance blocks unsafe launch | Zero balance shows add-funds path and read-only catalog context | Insufficient balance blocks launch before allocation mutation | Add funds or use admin/dev seeded balance; refresh usage | Stripe/auth/usage failures show product-owned error/correlation | Usage/audit rows prove billing stop after release | Billing/payments UX | Billing usage/balance, checkout/customer portal, admin finance | `v3-account-billing.test.tsx`, `auth-login.spec.ts`, `billing_launch_guard_uat.sh` | `UAT-BILLING-001` | Stripe/dev checkout config; low-balance account | kind/dev | L3 | needs_coverage | frontend/backend/finance | `HARNESS-FIX-BILLING-RETURN-LOW-BALANCE-UAT-001` |

## P1 Flow Matrix

| Flow ID | Persona | Entry Point | Happy Path | Empty State | Blocked State | Recovery | Negative Path | Cleanup | UX Anchor | Contract / CLI Anchor | Automation Anchor | UAT Row | Fixture / Env | Lane | Level | Status | Owner | Gap Task |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FLOW-APP-RUNTIME-001 | App user | `/apps`, `/launch/app/:slug`, `/workloads/:id?tab=connect` | Launch supported OCI app, open managed route, runtime state visible | Missing compute/storage/service account dependency can be created/selected inline | Precheck blocks missing dependency before submit | Inline dependency creators and access workbenches | Runtime/proxy/upstream failures classify user-safe route error | Decommission app and cleanup retained/non-retained fixtures | App workload journey; app shell extensibility | v3 app launch/precheck/runtime APIs; SDK contract | `v3-compute-apps.spec.ts`, `v3-live-app-runtime-smoke.spec.ts`, `app_sdk_live_launch_connect_decommission_smoke.sh` | `UAT-APP-001`, `UAT-APP-002` | One retained fixture per supported app; route readiness | kind/dev | L5 | ready_for_broad_uat | appplatform/backend/ops/frontend | `PSSM-UAT-APP-FIXTURE-ONE-EACH-001` closed |
| FLOW-APP-DEVELOPER-001 | App developer | `/developer/*`, `/apps/artifacts`, CLI/SDK | Developer finds docs/API/downloads, publishes or validates artifacts, uses service account | No service account/artifact links to create path | Artifact trust or service-account policy blocks with owner/action | Create service account; publish intent; verify/promote artifact | SDK/API errors preserve correlation and actor attribution | Retire/deprecate/revoke artifact | Developer docs/app builder flow | CLI command catalog; app artifact APIs; OpenAI-compatible endpoint | `developer-downloads.spec.ts`, `v3-app-artifacts-page.test.tsx`, app SDK smokes, `openai_sdk_api_bearer_smoke.py` | `UAT-APP-002`, `UAT-OPENAI-001` | Service account token; app artifact; OpenAI route | dev required, kind conditional | L4 | ready_for_broad_uat | appplatform/backend/security | none |
| FLOW-STORAGE-LIFECYCLE-001 | Project admin | `/storage`, `/launch/storage`, `/storage/:id` | Create bucket, browse details, upload/download/mkdir/delete object | No bucket shows launch/create path | Quota/provider/grant/mount blockers show owner and recovery | Create bucket; grant/mount where supported; retry object op | Path traversal/quota errors are structured | Delete bucket/object or retained fixture manifest | Storage journey | v3 storage APIs plus legacy storage CRUD | `v3-storage-access-account.spec.ts`, `v3-storage-page.test.tsx` | `UAT-STORAGE-001` | Bucket, grant/mount target, cleanup manifest | kind/dev | L3 | needs_coverage | storage/backend/frontend/ops | `UAT-BUG-STORAGE-MOUNT-GRANT-UAT-COVERAGE-001` |
| FLOW-SCHEDULER-RUNTIME-001 | End user / ops | `/schedulers`, app launch/detail routes | RKE2/Slurm scheduler launch/connect proves controller and workers | No controller/credential/worker capacity blocks before submit | Precheck and submit return same dependency blockers | Create/select bootstrap credential and target allocation | Runtime 500s become precheck/product errors | Decommission scheduler and workers; cleanup manifest | Scheduler/app workload journey | v3 scheduler shared-runtime APIs; app runtime operations | `v3-schedulers-page.test.tsx`, `v3-app-launch-config-parity.spec.ts`, `demo_scheduler_uat.sh` | `UAT-SCHED-001`, `UAT-SCHED-002` | Scheduler controller SSH/provider capacity | dev required, kind conditional | L4 | ready_for_broad_uat | backend/ops/appplatform | `PSSM-SCHEDULER-LAUNCH-PRECHECK-SUBMIT-PARITY-001` closed |
| FLOW-ADMIN-LIFECYCLE-001 | Platform operator | `/platform/lifecycle/*` | Inspect nodes/MAAS/onboarding/decommission, probe/recover safely | No sites/nodes shows onboarding/discovery path | Stale/no-heartbeat/provider drift blocks launchable state | Provider-node preflight, onboarding resume/rerun | Failed ops actions show evidence and correlation | Decommission/remove/detach with audit trail | Admin operations; V3 admin workbench model | v3 platform lifecycle APIs; MAAS admin APIs | `v3-admin-platform.spec.ts`, platform lifecycle component tests, provider preflight scripts | `UAT-OPS-001`, `UAT-OPS-002` | MAAS/proxmox profile, provider inventory | kind/dev | L5 | ready_for_broad_uat | ops/backend/frontend | none |
| FLOW-OBS-EVIDENCE-001 | Ops/security | `/platform/evidence/*`, `/platform/ops/*`, `/edge-error` | Query evidence/audit, follow observability pivots, prove user-safe errors | Empty logs/evidence show filters and next actions | Observability backend unavailable classified before UAT | Pre-UAT clean-log/user-safe-error gates | Branded error with correlation ID, classified 5xx owner | Evidence packet and cleanup manifest captured | Evidence/audit/ops journeys | Evidence/audit/ops read models; log/trace APIs | `v3-admin-platform.spec.ts`, `v3-audit-evidence-page.test.tsx`, `pre_uat_clean_log_gate.sh`, `pre_uat_user_safe_error_gate.sh` | `UAT-EDGE-001`, `UAT-OBS-001` | Loki/Tempo/Grafana reachability, correlation IDs | kind/dev | L5 | ready_for_broad_uat | ops/security/frontend | none |
| FLOW-HARNESS-PREUAT-001 | Governance / ops | `scripts/ci/pre_uat_readiness_gate.sh`, UAT package summaries | Pre-UAT gate reports readiness and full UAT summary maps every required row | Missing fixtures are blocked prerequisites, not product failures | Required gates fail strict mode before broad UAT starts | Create fixture/env follow-up with owner and expiry | Harness forced failure preserves body, exit status, cleanup continuation | Cleanup manifest validates retained/cleaned/blocked resources | Pre-UAT runbook; UAT matrix | Pre-UAT scripts; local UAT summary; cleanup manifest validator | `pre_uat_readiness_gate.sh`, `uat_harness_failure_injection.sh`, `local_uat_summary.sh`, `kind_smoke_profile.sh` | `UAT-HARNESS-001` | UAT artifacts, readiness JSON, cleanup manifest | kind/dev | L4 | needs_coverage | governance/ops | `HARNESS-FIX-FLOW-COVERAGE-SUMMARY-GATE-001` |

## P0/P1 Gap List

| Priority | Gap | Classification | Why It Matters | Owner | Gap Task | detected_by | expected_gate |
|---|---|---|---|---|---|---|---|
| P0 | UAT matrix closeout snapshot still says kind compute capacity blocks full UAT, but Fairway has later pass evidence for capacity and terminal stream recovery. | docs/runbook gap | Operators may make wrong go/no-go decisions from stale canonical docs. | Product Quality / ops docs | `DOC-FIX-UAT-MATRIX-SNAPSHOT-REFRESH-001` | `manual_review` | `review_gate` |
| P0 | UAT summaries must map required `UAT-*` rows to Product Quality flow IDs and required subpaths (happy, empty, blocked, recovery, negative, cleanup, fixture, environment). | harness gate / missing UAT row mapping | A green UAT can still hide missing empty-state, recovery, fixture, or environment coverage. | Governance / ops harness | `HARNESS-FIX-FLOW-COVERAGE-SUMMARY-GATE-001` | `manual_review` | `pre_uat_gate` |
| P1 | Storage has product/e2e CRUD coverage, but broad UAT evidence does not yet prove bucket grant/mount/detach lifecycle or approved skip for those subpaths. | missing UAT coverage / fixture setup | `UAT-STORAGE-001` requires create/grant/mount/detach/delete or approved skip. | Storage/backend/ops | `UAT-BUG-STORAGE-MOUNT-GRANT-UAT-COVERAGE-001` | `manual_review` | `pre_uat_gate` |
| P1 | Billing has unit/e2e smoke coverage, but broad UAT needs explicit low-balance persistent UX, Stripe return handling, and insufficient-balance guard evidence mapped to the flow row. | missing e2e/integration coverage | Billing safety is P0/P1 because it blocks unsafe allocation and finance handoff. | Finance/backend/frontend | `HARNESS-FIX-BILLING-RETURN-LOW-BALANCE-UAT-001` | `manual_review` | `pre_uat_gate` |
| P1 | Account MFA setup worked, but Product Flow Coverage did not model existing-factor manage/remove behavior, lost-phone recovery, provider return states, admin/break-glass policy, or user-facing AI Cloud branding before declaring MFA product-ready. | missing flow coverage / UX state coverage | Users with an existing authenticator saw a setup action advertised as manage; loss/recovery and disable/delete factor flows are core MFA UX and safety paths. | Product Quality / frontend / identity / security / ops | `PRODUCT-GAP-IAM-MFA-FACTOR-MANAGE-FLOW-001`; `PRODUCT-GAP-IAM-MFA-FACTOR-REMOVE-DISABLE-FLOW-001`; `PRODUCT-GAP-IAM-MFA-FACTOR-RECOVERY-FLOW-001`; `PRODUCT-GAP-IAM-MFA-ADMIN-BREAKGLASS-POLICY-FLOW-001`; `PRODUCT-GAP-IAM-MFA-PROVIDER-FACTOR-READBACK-001`; `UAT-BUG-IAM-MFA-PROVIDER-RETURN-FLOW-001`; `PRODUCT-GAP-IAM-MFA-USER-FACING-BRANDING-SCAN-001` | `manual_kind_uat` | `product_flow_gate` |

## Broad UAT Decision

Ready for broad UAT, subject to normal environment readiness:

- `FLOW-AUTH-SESSION-001`
- `FLOW-PROJECT-IAM-001`
- `FLOW-SSH-KEY-001`
- `FLOW-COMPUTE-LAUNCH-001`
- `FLOW-COMPUTE-RELEASE-001`
- `FLOW-TERMINAL-CONNECT-001`
- `FLOW-APP-RUNTIME-001`
- `FLOW-APP-DEVELOPER-001`
- `FLOW-SCHEDULER-RUNTIME-001`
- `FLOW-ADMIN-LIFECYCLE-001`
- `FLOW-OBS-EVIDENCE-001`

Ready with accepted gaps:

- `FLOW-ACCOUNT-MFA-001`: current kind/dev UI/UAT coverage is accepted for
  setup/add-first-factor posture, sessions, read-only Platform IAM readiness,
  and evidence pivots. It is not accepted as a completed existing-factor
  management, factor remove/disable, lost-phone recovery, locked-out recovery,
  provider return, provider unavailable, admin/ops/superadmin behavior,
  break-glass, non-human exclusion, branding, provider-internals, or
  current-session assurance flow until
  `MFA_Factor_Lifecycle_UAT_Coverage_v1.md` blockers are closed or explicitly
  deferred. Production readiness remains gated by
  `HARNESS-IAM-MFA-NONLIVE-CLAIM-CONTRACT-PROOF-001` and
  `IAM-MFA-SENSITIVE-OPS-GATE-001`; no page may claim local sensitive-operation
  MFA enforcement until claim proof is accepted.

Blocked or not ready for broad completion claims until task closure:

- `FLOW-BILLING-SAFETY-001`: needs
  `HARNESS-FIX-BILLING-RETURN-LOW-BALANCE-UAT-001`.
- `FLOW-STORAGE-LIFECYCLE-001`: needs
  `UAT-BUG-STORAGE-MOUNT-GRANT-UAT-COVERAGE-001`.
- `FLOW-HARNESS-PREUAT-001`: needs
  `HARNESS-FIX-FLOW-COVERAGE-SUMMARY-GATE-001`.

Documentation freshness blocker:

- Platform UAT snapshot needs
  `DOC-FIX-UAT-MATRIX-SNAPSHOT-REFRESH-001` before it is used as the sole
  go/no-go source for kind/dev broad UAT.

## Gate Rule For New User-Visible Work

Before a new user-visible workflow is marked ready:

1. Add or update one flow row in this matrix.
2. Link UX anchors and contract/API/CLI anchors.
3. Identify empty, blocked, recovery, negative, and cleanup paths.
4. Attach frontend e2e, API/CLI/integration smoke, or accepted-gap evidence.
5. Map to one or more `UAT-*` rows in
   `Platform_UAT_Completeness_Matrix_v1.md`.
6. Create a Fairway follow-up for any P0/P1 gap with `detected_by` and
   `expected_gate` classifiers.