Appearance
Q4 2026 Disaster Recovery Test Plan
Purpose: Second annual DR validation incorporating lessons from Q2 and expanding scope to areas deferred in Q2. Satisfies HIPAA §164.308(a)(7)(ii)(D) testing and revision requirement.
Owner: Rory (Security Officer) Test window: November 2026 — target Saturday the first full weekend after the SOC2 audit closes, 08:00 PT start, 12-hour window Participants: Rory (lead), Kevin, Greg, Adriana (comms observer), plus auditor observer if SOC2 team requests Prerequisites: Q2 test findings actioned; Q2 runbook revisions merged
Incorporating Q2 lessons
This plan is a living document. After the Q2 post-mortem (see q2-dr-test-findings-2026.md once it exists), every identified gap becomes a checklist row here. Representative carry-over items expected from Q2 (prospective, pending actual Q2 results):
- Any runbook ambiguity discovered during Q2 — validate the corrected language
- Any RTO/RPO miss — retest that scenario
- Any stale credential or connector issue — confirm rotation and re-test
- Any missing or slow evidence capture — re-validate after process improvement
The final version of this plan will be updated with concrete carry-overs within 30 days of the Q2 post-mortem.
Focus areas
Focus 1 — RSV restore path independent of Veeam
Why: Ransomware scenarios often target backup servers first. If Veeam is unavailable, the RSV restore path must be the fully-sufficient alternative.
Activities:
- Restore a full production-class VM (not a throwaway) from RSV to an isolated DR RG
- Boot the VM, verify application-level integrity
- Measure RTO for the end-to-end RSV restore including customer file share item-level restore
- Validate immutability (Unlocked) does not block legitimate restore under pressure
- Validate KMS key availability — Customer Managed Key must be accessible from DR subscription
Focus 2 — Updated IR runbook validation
Why: Runbook changes merged after Q2 must be validated under test conditions.
Activities:
- Dry-run of
runbooks/incident-response.mdRansomware section with the Q2 revisions applied - Dry-run of tabletop scenario (see
compliance/ir-tabletop-scenario-2026.md) — verify comms chain, decision authority, recovery sequence - Confirm every runbook step has a named owner and a verification artifact
Focus 3 — Backup restore validation against psql-secops-prod
Why: SecOps platform hosts all compliance evidence — its recoverability is itself an auditable control.
Activities:
- Point-in-time restore of psql-secops-prod to a new server in a DR RG
- Validate row counts in key tables (incidents, findings, evidence, changes, known_good)
- Run a read-only SecOps instance against the restored DB
- Measure RPO — delta between last committed row in production and restored row
- Verify pgaudit (if enabled by Q4) streams to LAW during the restore operation
Focus 4 — Palo Alto NVA rebuild from backup
Why: Not tested in Q2. If the firewall fleet is compromised or misconfigured, the rebuild path must be proven.
Activities:
- Stand up a new VM-Series from marketplace image in DR VNet
- Push Panorama configuration backup to new firewall
- Validate DNS Security license reattaches
- Validate App-ID and threat prevention are active
- Document RTO for NVA rebuild (new metric for Q4)
Focus 5 — DDE environment spot-check
Why: DDE is customer-facing (Medicare). Its DR is tested separately, but a spot-check here validates cross-tenant dependencies.
Activities:
- Confirm DDE tenant break-glass accounts usable
- Validate DDE Twingate connector
- Verify cross-tenant B2B guest flow still works if PROD Entra is degraded
Success criteria
| Metric | Target | How measured |
|---|---|---|
| RTO — psql-secops-prod PITR | ≤ 2 hours | Restore start to SecOps UI read-only |
| RTO — full production-class VM via RSV | ≤ 4 hours | RSV restore start to VM login |
| RTO — Palo Alto NVA rebuild | ≤ 6 hours (new metric) | Marketplace deploy to policy active |
| RPO — psql-secops-prod | ≤ 15 minutes | Delta measurement |
| Runbook clarity | 0 ambiguous steps | Post-mortem review |
| Gap carry-over from Q2 | 100% re-validated | Each Q2 gap marked closed or re-escalated |
Runbook outline
Identical phase structure to q2-dr-test-plan-2026.md:
- Pre-test (T-7 days and T-1 day)
- Execute (T+0 to T+12h)
- Validate (T+4 to T+8h)
- Failback (T+8 to T+11h)
- Post-mortem (T+7 days)
Detailed steps populated at T-14 once Q2 lessons are finalized.
Schedule
- November 2026 — test window (exact Saturday TBD once SOC2 audit closes)
- December 2026 — post-mortem + findings writeup, threat model annual review (see
security/threat-model/threat-model-2026.md) - January 2027 — Q1 action items drafted from findings, added to annual review checklist
Out of scope for Q4
- Failover of the live DDE published app to Medicare (requires Medicare coordination)
- Rebuild of Azure OpenAI service (recovery handled by Microsoft under MBSA)
Document history
| Date | Change | Author |
|---|---|---|
| April 2026 | Initial Q4 DR test plan — outline pending Q2 outcomes | Kobe |