Cluster troubleshooting: failover clusters and Hyper-V VMs

Clusters and hypervisors are built to survive failures, so when they do break, the cause is often hidden behind an automatic failover. Cluster troubleshooting means reading what the cluster saw, in order, across every node.

In this lesson you’ll check CLUSTER1’s health, generate and read the cluster log, run safe validation tests, read failover events, and check a Hyper-V VM’s health on your laptop. Then you’ll break a cluster IP resource and a VM’s memory setting, and fix both.

TimeYou needBy the end you can
About 90 minutesDC01, SRV01 (iSCSI target), NODE1 and NODE2 (CLUSTER1 from Lesson 37), and the Hyper-V host with HV-TEST01 from Lesson 28Find why a clustered role or a VM won’t come online, using cluster logs, events and Hyper-V logs

What it is, in one line

Cluster troubleshooting is finding which node, resource or dependency failed, and why, by reading cluster state, the cluster log and failover events in time order.

In Simple Words

In Simple Words

Think of two bus drivers sharing one route. If one feels sick, the other takes the bus and the passengers hardly notice. That’s failover.

But if the bus keeps changing drivers every hour, something is wrong: maybe a flat tyre, maybe a wrong ticket machine. The passengers only see delays; the depot’s logbook shows every handover and why.

The cluster log is that logbook. You read it to find what made the drivers swap.

Engineer's View

Engineer’s View

Structure: a cluster has nodes, networks, storage, a quorum witness and roles (groups). Each role holds resources (IP address, network name, disk, file server or VM) with dependencies: the network name needs its IP, the file server needs the name and the disk.

State: Get-ClusterNode, Get-ClusterGroup, Get-ClusterResource and Get-ClusterQuorum show what’s online, failed or partially online.

Evidence: Get-ClusterLog writes a detailed text log per node (times in UTC by default; add -UseLocalTime). The System log has FailoverClustering events: 1069 (resource failed), 1205 (role couldn’t come online), 1254 (failover threshold exceeded), 1135 (node removed), 1177 (quorum lost). Validation (Test-Cluster) checks configuration, but storage tests take disks offline, so never run them casually on production.

Hyper-V: VM state and status from Get-VM, integration services, the Hyper-V-VMMS and Hyper-V-Worker logs, checkpoint chains (avhdx files) and host resources. Common failures: not enough memory, missing or locked VHDs, broken checkpoints, and live migration blocked by CPU or configuration differences.

How it works, step by step

How a clustered role depends on its resources:

Cluster troubleshooting: resource dependencies in a clustered file server role

Resources come online from the bottom up. If the IP fails, the name can’t start, and the file server above it stays offline, so the first failed resource is the one that matters.

When a resource fails, the cluster restarts it on the same node, then moves the role to another node. If it keeps failing, the role stops trying after the failover threshold, and you see event 1254.

Always collect the cluster log from all nodes for the same time window. The node that reports the problem isn’t always the node that caused it.

Event, meaning and where to look next:

Cluster troubleshooting events 1069, 1205, 1135 and 1177 with next steps

In the lab you’ll:

  • Check nodes, roles, resources and quorum on CLUSTER1.
  • Generate and search the cluster log.
  • Run safe validation tests.
  • Read FailoverClustering events.
  • Check a Hyper-V VM’s health and logs on your laptop.

Hands-on lab: read the cluster and VM evidence

Start DC01, SRV01, NODE1 and NODE2, and make sure CLUSTER1 is online. Work as SAATLAB\adm.rahul in PowerShell as administrator on NODE1 unless a step says otherwise.

1Check nodes, roles, resources and quorum

On NODE1:

Get-ClusterNode -Cluster CLUSTER1 | Select-Object Name, State
Get-ClusterGroup -Cluster CLUSTER1 | Select-Object Name, OwnerNode, State
Get-ClusterResource -Cluster CLUSTER1 | Where-Object State -ne 'Online' | Select-Object Name, State, OwnerGroup
Get-ClusterQuorum -Cluster CLUSTER1 | Select-Object QuorumResource

You’ll see something like this:

Name  State
----  -----
NODE1 Up
NODE2 Up

Name              OwnerNode State
----              --------- -----
Available Storage NODE2     Online
Cluster Group     NODE1     Online
FS-CLUSTER        NODE1     Online

QuorumResource
--------------
Cluster Disk 2

Both nodes are up, every role is online, no resource is failed (the third command printed nothing) and the disk witness is in place. This is your healthy picture to compare against later.

✅ Checkpoint: Both nodes are Up, all groups are Online, and no resource is failed.

2Generate and search the cluster log

Collect the last 30 minutes from both nodes, in local time:

New-Item C:\Lab -ItemType Directory -Force | Out-Null
Get-ClusterLog -Cluster CLUSTER1 -Destination C:\Lab -TimeSpan 30 -UseLocalTime
Get-ChildItem C:\Lab\*cluster.log | Select-Object Name, Length
Select-String -Path C:\Lab\NODE1_cluster.log -Pattern ' ERR ' | Select-Object -Last 2 Line

You’ll see something like this:

Name               Length
----               ------
NODE1_cluster.log 3145728
NODE2_cluster.log 2988032

Line
----
00000f2c.00001a10::2026/10/20-11:02:14.512 ERR   [RES] Network Name <FS-CLUSTER>: Unable to update DNS, status 9017
00000f2c.00001a10::2026/10/20-11:02:44.530 ERR   [RES] Network Name <FS-CLUSTER>: Unable to update DNS, status 9017

The cluster log is noisy, so search for ERR and WARN lines, then read the lines around the time of the problem. Your file may have no errors at all, which is good news.

✅ Checkpoint: C:\Lab holds a cluster log for each node, and you can search them for ERR lines.

Cluster troubleshooting lab: generating and searching the cluster log
Cluster troubleshooting: one cluster log per node, searched for errors

3Run safe validation tests

Run the tests that don’t touch storage:

Test-Cluster -Cluster CLUSTER1 -Include 'Inventory', 'Network', 'System Configuration' -ReportName C:\Lab\Validate-NoStorage

You’ll see something like this:

WARNING: Network - Validate Network Communication: The test reported some warnings..
WARNING:
Test Result:
ClusterConditionallyApproved
Testing has completed successfully. The configuration appears to be suitable for clustering. However, you should
review the report because it may contain warnings which you should address to attain the highest availability.

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        20/10/2026     11:10         412544 Validate-NoStorage.htm

Open the .htm report and read every warning. In the lab, a warning about a single network path between nodes is normal. Storage tests take the shared disks offline, so on a real cluster run them only in a maintenance window.

✅ Checkpoint: Test-Cluster produced C:\Lab\Validate-NoStorage.htm without running storage tests.

4Read FailoverClustering events

On NODE1, list the latest cluster events from the System log:

Get-WinEvent -FilterHashtable @{ LogName = 'System'; ProviderName = 'Microsoft-Windows-FailoverClustering' } -MaxEvents 5 |
    Select-Object TimeCreated, Id, @{ n = 'Message'; e = { ($_.Message -split "\r?\n")[0] } } | Format-Table -Wrap

You’ll see something like this:

TimeCreated            Id Message
-----------            -- -------
20/10/2026 10:41:07  1135 Cluster node 'NODE1' was removed from the active failover cluster membership.
20/10/2026 10:40:58  1177 The Cluster service is shutting down because quorum was lost.

These are probably your own tests from Lesson 37. On a real cluster, 1135 points to network or node health, 1177 to quorum design, and 1069 or 1205 to a specific resource.

✅ Checkpoint: You can list recent FailoverClustering events and say what each ID means.

5Check a Hyper-V VM’s health

On your laptop (the Hyper-V host from Lesson 28), in PowerShell as administrator:

Get-VM HV-TEST01 | Select-Object Name, State, Status, Uptime, MemoryAssigned
Get-VMIntegrationService HV-TEST01 | Select-Object Name, Enabled, PrimaryStatusDescription
Get-WinEvent -LogName Microsoft-Windows-Hyper-V-VMMS-Admin -MaxEvents 3 |
    Select-Object TimeCreated, Id, @{ n = 'Message'; e = { ($_.Message -split "\r?\n")[0] } } | Format-Table -Wrap

You’ll see something like this:

Name      State   Status             Uptime   MemoryAssigned
----      -----   ------             ------   --------------
HV-TEST01 Running Operating normally 00:12:41     2147483648

Name                    Enabled PrimaryStatusDescription
----                    ------- ------------------------
Guest Service Interface False   OK
Heartbeat               True    OK
Key-Value Pair Exchange True    OK
Shutdown                True    OK
Time Synchronization    True    OK
VSS                     True    OK

“Operating normally” and Heartbeat OK mean the guest is alive and talking to the host. A VM that’s Running with a heartbeat of “Lost Communication” is usually hung inside. The VMMS log records why a VM failed to start, move or take a checkpoint.

✅ Checkpoint: HV-TEST01 shows Operating normally and its integration services report OK.

Break It, Then Fix It

Two failures that stop a role or a VM from coming online.

1Break it: give the file server role a duplicate IP

On NODE1, change FS-CLUSTER’s IP address to SRV01’s address, then restart the role:

$ip = Get-ClusterResource -Cluster CLUSTER1 | Where-Object { $_.OwnerGroup -eq 'FS-CLUSTER' -and $_.ResourceType -eq 'IP Address' }
$ip | Set-ClusterParameter -Name Address -Value 10.0.0.20
Stop-ClusterGroup -Cluster CLUSTER1 FS-CLUSTER | Out-Null
Start-ClusterGroup -Cluster CLUSTER1 FS-CLUSTER

You’ll see something like this:

Name       OwnerNode State
----       --------- -----
FS-CLUSTER NODE1     Failed

Find the first failed resource and the event:

Get-ClusterResource -Cluster CLUSTER1 | Where-Object OwnerGroup -eq 'FS-CLUSTER' | Select-Object Name, ResourceType, State
Get-WinEvent -FilterHashtable @{ LogName = 'System'; ProviderName = 'Microsoft-Windows-FailoverClustering'; Id = 1069, 1205 } -MaxEvents 2 |
    Select-Object Id, @{ n = 'Message'; e = { ($_.Message -split "\r?\n")[0] } } | Format-Table -Wrap

You’ll see something like this:

Name                 ResourceType  State
----                 ------------  -----
FS-CLUSTER           Network Name  Failed
FS-CLUSTER-Server    File Server   Offline
IP Address 10.0.0.46 IP Address    Failed
Cluster Disk 1       Physical Disk Online

  Id Message
  -- -------
1205 The Cluster service failed to bring clustered role 'FS-CLUSTER' completely online or offline.
1069 Cluster resource 'IP Address 10.0.0.46' of type 'IP Address' in clustered role 'FS-CLUSTER' failed.
Cluster troubleshooting break lab: a duplicate IP fails the whole role
The IP failed first, so the name and file server above it could not start

The resource name still shows the old address, but its Address parameter is now 10.0.0.20, which SRV01 already uses. The IP fails, so the name above it fails, and the file server never starts. Always fix the lowest failed resource first. [Add your screenshot here]

Fix it: put the right address back and start the role:

$ip | Set-ClusterParameter -Name Address -Value 10.0.0.46
Start-ClusterGroup -Cluster CLUSTER1 FS-CLUSTER

You’ll see something like this:

Name       OwnerNode State
----       --------- -----
FS-CLUSTER NODE1     Online

✅ Checkpoint: FS-CLUSTER failed with events 1069 and 1205 on the duplicate IP, and is online again with 10.0.0.46.

2Break it: a VM that asks for too much memory

On your laptop, stop HV-TEST01 and give it far more startup memory than the host has:

Stop-VM HV-TEST01
Set-VMMemory HV-TEST01 -DynamicMemoryEnabled $false -StartupBytes 64GB
Start-VM HV-TEST01

You’ll see something like this:

Start-VM : 'HV-TEST01' failed to start.
Not enough memory in the system to start the virtual machine HV-TEST01 with ram size 65536 megabytes.

Hyper-V reserves the full startup memory before the VM boots. If the host can’t give it, the VM won’t start, even if the guest would only use a fraction of it. The same thing happens after a failover when the surviving node doesn’t have enough free memory for every VM.

Fix it: set a sensible size and start the VM:

Set-VMMemory HV-TEST01 -StartupBytes 2GB
Start-VM HV-TEST01
Get-VM HV-TEST01 | Select-Object Name, State, MemoryAssigned

You’ll see something like this:

Name      State   MemoryAssigned
----      -----   --------------
HV-TEST01 Running     2147483648

✅ Checkpoint: HV-TEST01 failed to start with 64 GB and runs again with 2 GB.

This happened at work

The live migration that was refused

A team added two new Hyper-V hosts with newer processors to an existing cluster. Everything validated, but live migrating VMs from the new hosts back to the old ones failed. During a patch weekend, VMs on a new host couldn’t be moved, so its restart was delayed for hours.

The error said the hardware on the destination wasn’t compatible with the hardware requirements of the virtual machine. VMs that started on the newer CPUs were using instructions the older CPUs didn’t have.

The fix was processor compatibility mode. With each VM shut down once, Set-VMProcessor -CompatibilityForMigrationEnabled $true limits the CPU features the VM sees, so it can move between both generations.

The fix: compatibility mode on VMs in mixed clusters, and a rule to keep hosts in one cluster on the same CPU generation where possible.

The lesson: a cluster that validates can still have VMs that can’t move. Test live migration in every direction before you rely on it.

Common mistakes and troubleshooting

MistakeWhat happensWhat to do instead
Fixing the top resource firstIt fails againFind the lowest failed resource in the dependency chain
Collecting the cluster log from one node onlyYou miss the cause on the other nodeGet-ClusterLog for all nodes, same time window
Reading cluster log times as local timeEvents don’t line upUse -UseLocalTime, or remember it’s UTC
Running storage validation on productionShared disks go offlineOnly in a maintenance window, or skip storage tests
Deleting the cluster’s computer objects in ADRoles can’t come onlineProtect CNO and VCO objects (Lesson 37)
Filling a node with VMsFailover fails for lack of memoryKeep enough headroom for a node failure
Deleting avhdx files by handVM disk chain breaksRemove checkpoints through Hyper-V so they merge
Mixing CPU generations without a planLive migration refusedProcessor compatibility mode or matching hosts

Interview questions

Click a question to open it. Under each one you’ll find what the interviewer is really looking for, and one good way to answer.

Fresher level

1. What is a clustered role?

What they’re really checking: Do you know the basics?

How you could answer: A group of resources (IP, name, disk, service or VM) that fails over together between nodes.

2. How do you check if a cluster is healthy?

What they’re really checking: Do you know the commands?

How you could answer: Get-ClusterNode, Get-ClusterGroup, Get-ClusterResource and Get-ClusterQuorum, or Failover Cluster Manager.

3. What does a “Lost Communication” heartbeat on a VM mean?

What they’re really checking: Do you know Hyper-V basics?

How you could answer: The VM is running but the guest isn’t responding to the host, often because it’s hung or integration services failed.

2–5 years experience

4. A clustered role won’t come online. How do you troubleshoot?

What they’re really checking: Do you have a method?

How you could answer: List its resources, find the lowest failed one, read events 1069 and 1205, then the cluster log around that time, and fix that resource.

5. How do you get and read the cluster log?

What they’re really checking: Do you know the tool?

How you could answer: Get-ClusterLog -Destination with a -TimeSpan, ideally -UseLocalTime, then search for ERR and WARN around the failure time on all nodes.

6. Why might a VM fail to start after a failover?

What they’re really checking: Do you think about capacity?

How you could answer: Not enough memory on the surviving node, storage paths missing on that node, or a virtual switch with a different name.

Senior level

7. A cluster keeps losing a node every few days. What do you check?

What they’re really checking: Can you go deep?

How you could answer: Event 1135 times, cluster network health, NIC drivers and firmware, heartbeat thresholds, antivirus exclusions, and the cluster log on both nodes at those times.

8. Live migration fails between some hosts. Why?

What they’re really checking: Do you know migration requirements?

How you could answer: CPU differences (use compatibility mode), different virtual switch names, missing storage access, authentication settings or configuration version.

9. How do you design a cluster to be easy to troubleshoot?

What they’re really checking: Do you think ahead?

How you could answer: Consistent hardware and names, documented dependencies, a witness, monitoring of key events, regular validation reports and tested failover.

Quick quiz

Q1. Which resource should you fix first?

  1. The file server
  2. The lowest failed resource in the chain
  3. The witness
Show answer

B — Higher resources depend on it.

Q2. What does event 1069 mean?

  1. A node joined
  2. A cluster resource failed
  3. Quorum lost
Show answer

B — 1205 follows for the role.

Q3. Why use -UseLocalTime with Get-ClusterLog?

  1. It’s faster
  2. The log is in UTC by default
  3. It adds errors
Show answer

B — So times match other logs.

Q4. Why did HV-TEST01 fail to start in Break It step 2?

  1. Broken disk
  2. Not enough host memory for its startup RAM
  3. Wrong switch
Show answer

B — Hyper-V reserves startup memory.

Q5. What fixed live migration in the work story?

  1. A new cluster
  2. Processor compatibility mode
  3. More memory
Show answer

B — It hides newer CPU features.

Quick recap

  • Cluster troubleshooting starts with state: nodes, roles, resources and quorum.
  • Fix the lowest failed resource in the dependency chain first.
  • Get-ClusterLog -UseLocalTime from all nodes, plus events 1069, 1205, 1135 and 1177, tell the story.
  • Run validation without storage tests on live clusters.
  • For VMs, check state, heartbeat, the VMMS log, host memory and CPU compatibility.

Next lesson

Lesson 52: Sysinternals toolkit → You’ll use Process Explorer, Process Monitor, Autoruns, TCPView and other Sysinternals tools to see what Windows is really doing.

Further reading: cluster troubleshooting

Want to go deeper into cluster troubleshooting? Microsoft’s Get-ClusterLog reference explains every option, and the high availability troubleshooting docs cover common cluster failures. You can also see the full plan for this course on the Windows Server Administration course page.

Scroll to Top