Veeam Console — Day-to-Day Operations

Scope

This document covers daily Veeam Backup & Replication console operations for Cirius Group. It does not cover restores (see Veeam Restore Procedure), backup architecture (see Backup Architecture), or backup verification testing (see Backup Restore Verification).

Tape/archive is not in use at Cirius — Veeam targets S3 exclusively.

---

1 — Accessing the Veeam Console

Where Veeam Runs

Veeam Backup & Replication is installed on a Windows Server VM in the Azure Production subscription. The Veeam server holds the backup catalog, job configuration, and proxy settings. All management is done from this server — Veeam has no web UI, only the Windows desktop application.

Connecting

1. Connect via Twingate (required — the Veeam server has no public endpoint)
2. RDP to the Veeam server (hostname and IP in Keeper under "Veeam Server")
3. Credentials: Keeper → Veeam → Veeam Server Local Admin — use the local administrator account, not a domain account, unless domain accounts have been granted explicit Veeam Backup Administrator role
4. Once on the desktop, open Veeam Backup & Replication Console from the Start menu or the desktop shortcut
5. The console connects to the local Veeam Backup Service automatically — you do not need to specify a server address when running from the Veeam server itself

If you need to connect remotely from your own machine rather than RDP'ing to the server: install the Veeam Backup & Replication Console (client only) and connect to the Veeam server's hostname or IP over port 9392.

AWS Target Access

The Veeam jobs write to S3 in the AWS Backup account (863609217450). Veeam uses an IAM service account configured in the repository settings — you do not need direct AWS console access for routine console operations. IAM credentials for the Veeam S3 integration are stored in Keeper under "Veeam AWS IAM".

---

2 — Morning Job Status Check

What to Check First

When you open the console or start the morning review, go to:

Home (left sidebar) → Last 24 Hours section at the top of the dashboard

This shows all jobs that ran in the last 24 hours with color-coded status:

Color	Status	Meaning
Green	Success	Job completed with no issues
Yellow	Warning	Job completed but encountered non-critical issues
Red	Failed	Job did not complete — data was not backed up
Grey	Running	Job is currently executing
Blue	Idle	Job is scheduled but has not run yet

For a more detailed view:

Home → Jobs → Backup Copy (for the Azure → AWS Veeam replication jobs)

Click any job to see the last run summary: start time, end time, data processed, data transferred, status.

Acceptable Morning State

All nightly jobs should show green (Success). The expected backup window closes before business hours — if a job is still running (grey) at 8 AM PT, it is running long and should be investigated.

A yellow (Warning) state does not mean data was lost — it means the job completed but logged at least one non-fatal issue. See §4 for how to assess warnings.

A red (Failed) state means backup did not complete. See §3 for investigation steps.

Success vs. "Backed Up with Warnings"

Veeam distinguishes these in the job log:

• "Backed up successfully" — clean run, no issues. Nothing to do.
• "Backed up with warnings" — job finished and data is protected, but Veeam encountered something worth noting. You must review the warning to determine if action is needed (§4).
• "Failed" — job aborted. Data from this VM was not backed up in this run. RPO is now extended by one cycle. Investigate immediately (§3).

PowerShell Alternative

# Connect to the local Veeam server
Connect-VBRServer -Server localhost

# Get all session results from the last 24 hours
Get-VBRBackupSession | Where-Object {$_.CreationTime -gt (Get-Date).AddHours(-24)} |
  Select-Object Name, JobType, Result, CreationTime, EndTime |
  Format-Table -AutoSize

Results show: Success, Warning, or Failed per session.

---

3 — Investigating a Failed Job

Finding the Detailed Log

1. Home → Last 24 Hours → click the failed job (red)
2. In the job summary pane, click Statistics — this shows a per-VM breakdown with individual task statuses
3. Click the failed VM entry → Show Logs — the full task log opens with timestamped entries
4. Scroll to the first [error] line — that is the root cause, not the cascade of failures that follow it

Alternatively: History (left sidebar) → Backup Copy → find the failed session → right-click → Statistics

Common Failure Reasons and Fixes

Network timeout / connectivity to AWS S3

Log text: Error: Unable to connect to the target repository, Timed out, Connection refused

Fix:

1. Verify the Veeam server can reach S3: open PowerShell on the Veeam server and run Test-NetConnection s3.amazonaws.com -Port 443 — should show TcpTestSucceeded: True
2. Check the IAM credentials in the repository config: Backup Infrastructure → Backup Repositories → [repo name] → Edit → Credentials Verify the access key and secret are current (rotate in Keeper and update here if expired)
3. Check if AWS is experiencing an S3 outage in us-east-1 (the bucket region): visit https://health.aws.amazon.com/
4. Once connectivity is restored, retry the job (§6)

VSS error (Volume Shadow Copy Service)

Log text: Error: Failed to create VSS snapshot, VSS writer is in failed state, Snapshot creation failed

VSS failures happen on the source VM — the Windows VSS subsystem on the VM could not create a consistent snapshot.

Fix:

1. RDP to the affected source VM
2. Check VSS writer status: open an elevated command prompt and run vssadmin list writers — look for any writer in State: [7] Failed or Last error: Non-retryable error
3. Restart the failed VSS writer's service. Common ones:
   • SQL Server VSS writer: restart SQLWriter service
   • Exchange VSS writer (if applicable): restart MSExchangeIS
   • For an unknown writer, restart VSS and VolumeSnapshotService services
4. Run vssadmin list writers again — all writers should be in State: [1] Stable
5. Retry the Veeam job (§6)

Source VM snapshot issue

Log text: Failed to consolidate VM, Snapshot is too large, Snapshot limit reached, Too many snapshots

Azure VMs can accumulate snapshots from multiple sources (Veeam, RSV, manual). If snapshots pile up, subsequent snapshot operations fail.

Fix:

1. In the Azure Portal, go to the source VM → Disks → check for existing snapshots
2. If there are unexpected snapshots (not created by the current Veeam run), review their origin — do not delete RSV-managed snapshots or Veeam-in-progress snapshots
3. Stale manual snapshots can be deleted if confirmed not needed
4. Retry the Veeam job after clearing the snapshot issue

Target storage full (S3 capacity)

Log text: Insufficient storage space on the backup repository, Repository is out of disk space

This is unlikely with S3 (capacity is not a hard limit the same way disk is), but can occur if the bucket's lifecycle policy is misconfigured or Object Lock has prevented cleanup of old backups.

Important: The S3 bucket uses Object Lock in COMPLIANCE mode. Objects cannot be deleted by anyone — including Veeam — until their retention period expires. You cannot free space by deleting old backups manually. See §5 for how to handle this.

---

4 — Investigating a Warning

Finding Warning Details

Same path as failed jobs: Home → Last 24 Hours → click the yellow job → Statistics → click the VM → Show Logs — warnings appear as [warning] lines.

Warnings That Are Acceptable (No Action Required)

Warning	Reason	Safe to ignore?
Locked or open files skipped	Files in use at snapshot time (user temp files, browser cache, spool files)	Yes — these are not application data. Log it and move on.
Pagefile.sys / swapfile.sys excluded	Veeam skips page files by default — they are not useful in a backup	Yes — expected behavior
GuestHelper agent could not be contacted	Veeam's in-guest agent was unreachable, so Veeam fell back to a crash-consistent snapshot	Generally yes, but investigate if persistent across multiple runs — may indicate the Veeam agent needs reinstall

Warnings That Require Action

Warning	Reason	Action
High change rate detected	The VM changed much more data than baseline — large incremental, possibly a bulk write, data corruption event, or ransomware encryption in progress	Investigate the VM. Check what process drove the high change rate. If ransomware is suspected, escalate to incident response immediately.
RPO breach warning	The job took so long that RPO is approaching or exceeded	Review job duration trends. If a single run pushed RPO past 24 hours, the backup window needs tuning or the VM needs a proxy closer to source.
Backup chain length warning	The incremental chain is growing long and performance is degrading	Schedule an active full backup to reset the chain: right-click job → Active Full Backup
Repository approaching capacity threshold	Veeam's configured threshold for the repository is close to being hit	See §5 — S3 with Object Lock has specific constraints

---

5 — Repository Status and Space Management

Checking Repository Free Space

Backup Infrastructure (left sidebar) → Backup Repositories → select the S3 repository → the right pane shows Used Space and Free Space

For a more detailed view, right-click the repository → Properties — shows bucket name, capacity limits, and current utilization.

PowerShell:

Connect-VBRServer -Server localhost
Get-VBRBackupRepository | Select-Object Name, Path,
  @{N="UsedSpaceGB"; E={[math]::Round($_.Info.UsedSpaceGB, 2)}},
  @{N="FreeSpaceGB";  E={[math]::Round($_.Info.FreeSpaceGB, 2)}} |
  Format-Table -AutoSize

When Space Is Low — The S3 Immutability Constraint

The Veeam S3 bucket uses S3 Object Lock in COMPLIANCE mode with a 30-day default retention. This is a hard constraint:

• Objects within their retention window cannot be deleted by anyone — not by Veeam, not by the AWS root account, not by anyone
• You cannot free space by deleting old backups until their Object Lock retention expires naturally
• Reducing retention policy in Veeam does not retroactively unlock already-locked objects

What you can do when space is low:

1. Wait for natural expiration — objects older than 30 days (or whatever the retention is set to) will expire and become deletable. Veeam's GFS (Grandfather- Father-Son) retention handles cleanup automatically once locks expire.

2. Expand S3 capacity — S3 is not a fixed-size volume. "Space" in Veeam's repository view reflects configured capacity limits, not S3's actual capacity. If the configured limit is too tight, increase it in the repository settings: Backup Infrastructure → Backup Repositories → [repo] → Edit → Storage

3. Add a second repository — if the current bucket is genuinely exhausted, add a second S3 bucket as a scale-out backup repository: Backup Infrastructure → Scale-out Backup Repositories → Add This requires Rory's approval — it affects the DR architecture.

4. Never attempt to manually delete objects from S3 to free space for Veeam. Even if the Object Lock allows deletion (retention expired), manually deleting backup objects outside Veeam's control corrupts the backup chain. Let Veeam manage its own repository.

---

6 — Job Management

Starting a Manual Job Run

Useful when you need to run a job outside its schedule — after fixing a failure, before a planned VM migration, or for an ad-hoc backup.

Home → Jobs → Backup Copy → right-click the job → Start

The job runs immediately. Monitor progress in Home → Running Jobs or the job's Statistics pane.

PowerShell:

Connect-VBRServer -Server localhost
$job = Get-VBRJob -Name "Veeam Azure to AWS Nightly"
Start-VBRJob -Job $job

Disabling a Job Temporarily

Use this when a source VM is being migrated, decommissioned, or intentionally taken offline for maintenance. Disabling prevents the job from running on schedule.

Home → Jobs → Backup Copy → right-click the job → Disable

The job turns grey and will not run until re-enabled. The existing backup chain is preserved.

Note: disabling a job extends RPO for all VMs in that job. If disabling for more than 24 hours, document the maintenance window and notify Rory — HIPAA requires documented justification for any RPO gap.

Re-enabling a Job

Home → Jobs → Backup Copy → right-click the disabled job → Enable

The job resumes on its normal schedule. If you want it to run immediately after re-enabling, right-click → Start after enabling.

Changing Job Schedule

Home → Jobs → Backup Copy → right-click → Edit → advance through the wizard to the Schedule step.

Options: daily at a specific time, after another job completes, or manual-only (no schedule). The current nightly schedule runs at a time configured to complete before business hours start at 8 AM PT.

PowerShell (view current schedule):

Connect-VBRServer -Server localhost
$job = Get-VBRJob -Name "Veeam Azure to AWS Nightly"
$job.ScheduleOptions

Schedule changes require Rory's approval if they affect the HIPAA-documented backup window.

Active Full Backup (Reset the Incremental Chain)

If the incremental chain is long and degrading performance, or after a major VM change, run an active full:

Home → Jobs → Backup Copy → right-click → Active Full Backup

This creates a new full backup and starts a fresh incremental chain. It transfers significantly more data than a normal run and will take longer. Do not run during business hours unless necessary.

---

7 — Agent Policies (Agent-Based Backup)

When Agent Backup Is Used

Veeam's agentless backup (hypervisor snapshot) covers Azure VMs directly. Veeam agents are used for workloads that cannot be snapshot at the hypervisor level — typically:

• Physical servers
• VMs in environments where Veeam does not have hypervisor-level access
• Workloads requiring application-aware backup with granular control

Checking Agent Status

Inventory (left sidebar) → Physical Infrastructure → expand to find managed computers. Each computer shows its agent status:

Status	Meaning
Managed	Agent is installed, connected, and policy is applied
Unmanaged	Computer is discovered but no agent is installed
Protection error	Agent is installed but has an issue — click to see details
Offline	Agent is installed but not checking in

For jobs driven by agent policy: Home → Jobs → Agent Backup — same status colors as other job types.

Re-deploying an Agent

If an agent shows Protection error or is Offline:

1. Verify the source computer is online and reachable from the Veeam server (Twingate)
2. Inventory → Physical Infrastructure → right-click the computer → Rescan — this forces Veeam to re-probe the agent status
3. If the agent is corrupted or missing: right-click → Install Agent — Veeam pushes the agent installer over the network (requires admin credentials to the target, stored in the agent policy's credentials)
4. If credentials have changed: right-click the computer → Properties → update the credentials
5. After agent reinstall, verify the backup job picks up the computer on the next run: right-click the job → Start to test immediately

PowerShell:

Connect-VBRServer -Server localhost
# List all agent-protected computers and their status
Get-VBRDiscoveredComputer | Select-Object Name, IP, Status | Format-Table -AutoSize

---

8 — Reports

Daily Backup Report

Veeam sends an email completion report after each nightly job run. This is the primary daily touchpoint — if the email arrives and shows all green, no console access is needed.

If you need to view or regenerate reports from the console:

Reports (left sidebar) → Backup → select the report type:

• Backup Job Report — per-job detail: VMs processed, duration, data sizes, any errors
• Protected VMs — overview of all VMs and their last backup status

To generate a report manually: right-click the report → Generate → choose the date range. Reports can be exported as HTML or PDF from the report viewer.

Exporting for HIPAA Compliance Evidence

For HIPAA evidence (§164.308(a)(7)(ii)(A) — backup plan with testing evidence):

1. Reports → Backup → Backup Job Report
2. Set the date range to the audit period (typically the past month or quarter)
3. Right-click → Generate
4. In the report viewer: File → Export → PDF
5. Save to the compliance evidence folder or upload to SharePoint (Adriana manages the SharePoint compliance library)

The Veeam email reports are also archived — the ops-automation HIPAA audit email references Veeam job completion. Both the Veeam exports and the HIPAA audit emails count as backup evidence.

PowerShell (export job session history to CSV for audit):

Connect-VBRServer -Server localhost
$start = (Get-Date).AddDays(-30)
Get-VBRBackupSession | Where-Object {$_.CreationTime -gt $start} |
  Select-Object Name, JobType, Result, CreationTime, EndTime,
    @{N="DurationMin"; E={[math]::Round(($_.EndTime - $_.CreationTime).TotalMinutes, 1)}} |
  Export-Csv -Path "C:\VeeamReport-$(Get-Date -Format 'yyyy-MM').csv" -NoTypeInformation

---

9 — Common Operational Tasks

Adding a New VM to an Existing Job

When a new VM is provisioned and needs to be included in the nightly Veeam run:

1. Home → Jobs → Backup Copy → right-click the job → Edit
2. Advance to the Virtual Machines step
3. Click Add → From Infrastructure → browse to the Azure subscription and select the new VM
4. Complete the wizard — no other settings typically need changing
5. Click Finish — the VM is included in the next scheduled run
6. To immediately verify Veeam can see and access the VM, right-click the job → Start and monitor the first run

PowerShell:

Connect-VBRServer -Server localhost
$job = Get-VBRJob -Name "Veeam Azure to AWS Nightly"
$vm = Find-VBRViEntity -Name "vm-name-here"
Add-VBRViJobObject -Job $job -Entities $vm

Excluding a Disk from Backup

If a VM has a data disk that should not be backed up (e.g., a scratch disk with ephemeral data, or a disk with data excluded from HIPAA scope):

1. Home → Jobs → Backup Copy → right-click → Edit
2. Virtual Machines step → click the VM → Exclusions
3. Select the disk(s) to exclude by disk number (disk 0 = OS disk)
4. Finish and save

Note: excluding a disk from backup means that disk is not recoverable from Veeam. Only exclude disks explicitly confirmed to contain no ePHI and no critical data. Document the exclusion.

Adjusting Retention Policy

HIPAA requirement: do not reduce retention without Rory's explicit approval. Reducing retention means fewer recovery points and a potentially shorter window for detecting and recovering from a breach.

To view current retention settings:

Home → Jobs → Backup Copy → right-click → Edit → advance to the Storage step.

The retention is configured as a count of restore points or days, combined with GFS (Grandfather-Father-Son) policy for weekly, monthly, and yearly restore points.

To change:

• Increasing retention — safe, proceed after noting the change. Will consume more S3 storage over time.
• Decreasing retention — requires Rory's approval. Document the business justification. Note that due to S3 Object Lock, already-locked restore points will remain accessible until their Object Lock expiration even if removed from Veeam's retention policy — they will eventually be deleted by Object Lock expiration, not sooner.

---

Break-Glass: If Veeam Is Unavailable

If the Veeam server is down, unreachable, or the Veeam service is not running, Azure Recovery Services Vault is the fallback for VM restoration within Azure. RSV operates independently of Veeam — it does not require the Veeam server to be online.

For file, VM, or database recovery when Veeam is unavailable: see Veeam Restore Procedure §1 — Azure Recovery Services Vault.

For DR failover to AWS when both Azure and Veeam are unavailable: the Veeam backup data already written to S3 is accessible directly. The Veeam server is not required to read the S3 data — an EC2 instance can be built from the backup templates without the Veeam console. See Veeam Restore Procedure §2.

To restore the Veeam service itself: RDP to the Veeam server, check Services.msc for Veeam Backup Service — start it if stopped. If the VM itself is down, restart it from the Azure Portal. If the VM is corrupted, it can be restored from RSV (rsv-prod-backup covers the Veeam server VM like all other Production VMs).

---

Quick Reference

Task	Console Path
Check last 24h job status	Home → Last 24 Hours
View job details / logs	Home → Jobs → Backup Copy → click job → Statistics
Run a job now	Home → Jobs → right-click → Start
Disable a job	Home → Jobs → right-click → Disable
Active full backup	Home → Jobs → right-click → Active Full Backup
Edit job (add VM, change schedule)	Home → Jobs → right-click → Edit
Check repository space	Backup Infrastructure → Backup Repositories
Check agent status	Inventory → Physical Infrastructure
Generate report	Reports → Backup → Backup Job Report

Topic	Document
Restore from backup	Veeam Restore Procedure
Backup architecture, RPO/RTO	Backup Architecture
Quarterly verification testing	Backup Restore Verification
S3 Object Lock verification	Backup Restore Verification §3

---

Document History

Date	Change	Author
May 2026	Initial draft — day-to-day console operations: console access, morning check, failure investigation, warning assessment, repository management, job management, agent policies, reports, common tasks	Rory