# System health — Phase 6 milestone 10 Administrators can open **System health** on web or native mobile to inspect incoming email, AI extraction, incoming WhatsApp and mobile push. `GET /v1/system-health` requires a current active administrator session and returns a non-cacheable observation snapshot. Other staff cannot fetch it through either client or the API. This is read-only failure visibility. Refreshing does not call providers, start workers, retry jobs, send messages, change configuration or mark failures resolved. Milestone 10 required no migration; milestone 11 adds worker evidence tables, described in WORKER-HEALTH.md. The screens clear previous observations on refresh/error and sign out expired sessions. ## Reading the observations - **Disabled:** the server feature flag is off. Previously stored jobs/failures remain visible. - **Setup required:** required configuration or initialization is absent. The server returns a curated explanation, never credentials or private configuration values. - **Paused:** the incoming connector has been paused. - **Review needed:** a recorded failure, configuration/owner mismatch, expired processing lease or delayed work warrants investigation. This does not prove an ongoing provider outage. - **No issue detected in stored state:** no configured rule matched. This does not certify provider connectivity, worker liveness, signing, delivery or AI quality. Email checks the primary mailbox identity, current administrator owner, pause state, error/failure state and expired lease. An enabled, configured, initialized mailbox without a valid successful sync in the last 15 minutes needs review. Future or malformed timestamps cannot establish freshness. Last successful sync is displayed separately; outbound SMTP is not checked. AI extraction shows queued/running/failed jobs. A queued job last updated at least 15 minutes ago, a running lease expiring at or before the snapshot time, or a stored failed job needs review when enabled/configured. These counts do not measure assistant availability or model accuracy. WhatsApp reuses the existing connection configuration, identity and owner rules. Recorded conflicts since activation need review. No message arriving for a long time is **not** classified as an outage. The last received timestamp is informational; webhooks are not probed. Push shows queued jobs, pending receipts, due work delayed at least 15 minutes, expired sending claims, stored failures and unknown outcomes. Delays use `available_at`, so scheduled retries/receipt waits are not flagged before they are due. Terminal failed/unknown counts can include historical records. Unknown sends must not be automatically retried. No device tokens, tickets, user IDs, source IDs, raw errors, messages, model names or connection identifiers are returned. Queue counts use one SQL aggregation statement; connector state and ownership are read separately in the same request. The timestamp describes the observation request, not a continuous health guarantee. The screen has explicit refresh rather than background polling. There is no overall green readiness score. ## Operator follow-up Use existing Mailbox/WhatsApp controls and extraction review in Inbox. For push, use the private `scripts/inspect-push.py` procedure in PUSH.md and worker logs. Configuration changes, credential setup, activation and retries retain their existing review requirements. Do not automatically reactivate disabled integrations to remove a warning. Milestone 11 adds durable worker activity and bounded incident history, including empty-queue pass evidence. See WORKER-HEALTH.md. Hosted monitoring, external alert delivery, escalation recipients/thresholds and actual outage drills remain pending deployment. This milestone does not install scheduled monitoring or deliver alerts. ## Validation Backend tests cover administrator-only access, revoked sessions, non-cacheable responses, read-only SQL, secret-free responses, exact 15-minute/lease boundaries, fresh counts, mailbox configuration/owner/failure cases, malformed/future timestamps and idle/conflicting WhatsApp records. The same tests run against SQLite and isolated PostgreSQL 17. Browser tests exercise the real authenticated proxy, desktop/phone layout, refresh, clearing on failures/permission loss, expired sessions and denied trader/viewer access. TypeScript, shared contracts, web build and iOS/Android bundle export are checked separately. Native physical-device tests and hosted failure drills remain pending. Upstream mobile-tooling security issues remain documented in DEPENDENCY-REVIEW.md. Validated locally on 2026-10-10: 21 SQLite API tests (health plus operations), 13 PostgreSQL health cases, seven relevant browser cases, 21 shared contract tests, both TypeScript checks, production web build, and iOS/Android bundle exports passed. Desktop and phone-width web screenshots were inspected. Phase 6 milestone 12 adds explicit administrator event acknowledgement and private review notes. The health snapshot remains read-only; acknowledgement uses a separate POST and never changes worker health. See [WORKER-HEALTH.md](WORKER-HEALTH.md) for review and retention semantics. ## Outbound email setup Phase 3 milestone 10 adds a separate [private outbound configuration report](OUTBOUND-PREFLIGHT.md) beneath worker observations. It inspects an operator-selected private file and returns only fixed, redacted check results to administrators. Offline success never verifies authentication, delivery or sender ownership, and sending remains unavailable. No credential editor or connection test is exposed.