Post-Cleanup Drift Verification Runbook

Status: Runbook Owner: Infrastructure Trigger: Immediately after the June 2026 decommission wave completes, and quarterly thereafter Related: ciriussvcs-vnet-dependencies.md, wrkbizap-migration-plan.md

Purpose

Confirm that everything tagged Lifecycle=decommission has been removed from Azure, AWS, Terraform state, CMDB, and downstream dependencies (NSGs, firewall rules, DNS, monitoring) — with zero orphans left behind.

Prerequisites

• aztfexport installed on operator workstation
• Read access to all Azure subscriptions (both tenants) and AWS accounts
• SecOps platform reachable for CMDB queries
• az and aws CLIs authenticated

Procedure

1. Azure — aztfexport scan

For each subscription (Main, Logging, Billings, Firewall, DMZ, Prod, Dev, Identity across both tenants):

aztfexport subscription --subscription <sub_id> --non-interactive --output-dir /tmp/aztfexport-<sub_name>

Compare against Terraform-managed state:

cd /home/ubuntu/src/azure-infra/<stack>
terraform state list > /tmp/tfstate-<stack>.txt

Flag any resource in the aztfexport output that is not in Terraform state AND not explicitly marked as out-of-scope.

2. Azure — Lifecycle tag scan

az resource list --query "[?tags.Lifecycle=='decommission'].{name:name, rg:resourceGroup, type:type, sub:subscriptionId}" -o table

Expected: zero rows. Any row = orphan — open ticket to delete.

3. AWS — Resource Groups Tagging API

aws resourcegroupstaggingapi get-resources --tag-filters Key=Lifecycle,Values=decommission --query 'ResourceTagMappingList[].ResourceARN' --output text

Run per account. Expected: zero rows.

4. CMDB reconciliation

curl -s -H "X-API-Key: $SECOPS_API_KEY" https://soc.bedrockcybersecurity.org/api/cmdb/systems?status=active \
  | jq -r '.[] | select(.hostname | test("^(ACTDIRAP04|ACTDIRAP05|WRKBIZAP2[5-7])"))'

Expected: empty array. Any match = CMDB not updated, fix before close-out.

5. NSG rule scan — no references to removed CIDRs

for sub in $(az account list --query "[].id" -o tsv); do
  az network nsg list --subscription $sub -o json \
    | python3 -c "import sys,json; d=json.load(sys.stdin); [print(n['name'], r['name'], r.get('sourceAddressPrefix','')) for n in d for r in n.get('securityRules',[]) if '10.4.' in (r.get('sourceAddressPrefix') or '')]"
done

Expected: no output. Any hit = dangling NSG rule.

6. Palo Alto rule scan

Review Panorama for any rule whose source/destination zone references a decommissioned CIDR:

• 10.4.0.0/16 (CiriusSvcsVNET)
• Any other decommissioned spoke CIDRs in scope for the wave

Use the pan_analyzer output from ops-automation/.github/workflows/palo-alto-quarterly-bpa.yml.

7. DNS scan

Get-DnsServerResourceRecord -ZoneName ciriusgroup.com -RRType A | Where-Object {$_.RecordData.IPv4Address -like '10.4.*'}

Expected: no rows.

8. Terraform state drift

cd /home/ubuntu/src/azure-infra/<stack>
terraform init -backend=true -reconfigure
terraform plan -detailed-exitcode

Exit code 0 = clean. Exit code 2 = drift.

9. Sign-Off

curl -s -X POST -H "X-API-Key: $SECOPS_API_KEY" -H "Content-Type: application/json" \
  -d '{"evidence_type":"post-decommission-drift-verification","period_start":"<ISO>","period_end":"<ISO>","s3_key":"<bucket/key>","source_system":"PROD"}' \
  https://soc.bedrockcybersecurity.org/api/evidence

Sign-off requires Rory's review of the aggregated report (steps 1-8 combined into a Markdown document under bedrock-docs/compliance/).

Success Criteria

• [ ] Zero Lifecycle=decommission resources remain in any Azure subscription
• [ ] Zero Lifecycle=decommission resources remain in any AWS account
• [ ] Zero CMDB systems marked active reference decommissioned hostnames
• [ ] Zero NSG rules reference decommissioned CIDRs
• [ ] Zero Palo Alto rules reference decommissioned zones / CIDRs
• [ ] Zero DNS records point at decommissioned IPs
• [ ] All Terraform plan runs return exit code 0 (no drift)
• [ ] Evidence uploaded to S3 and registered in SecOps
• [ ] Rory has signed off the final report

Schedule

• Initial run: within 7 days of June 2026 decommission completion
• Quarterly thereafter: January, April, July, October — first business day

Change Log

Date	Change	Author
2026-04-16	Initial runbook	Kobe