Azure VM won’t boot: Boot diagnostics and Serial Console

Break 1 is quick; break 2 is the real incident.

1Break it: no boot diagnostics when you need it

A hardening script turns boot diagnostics off “to save storage”. Then someone tries the console:

az vm boot-diagnostics disable --resource-group rg-saat-l60 --name vm-boot -o none
az serial-console connect --resource-group rg-saat-l60 --name vm-boot

The CLI stops with an error saying the Serial Console needs boot diagnostics turned on.

The exact wording changes between CLI versions. The point is the same: with boot diagnostics off, you’re blind exactly when you need to see. That’s why Azure Policy has built-in definitions to require it.

Fix it: turn it back on with managed storage:

az vm boot-diagnostics enable --resource-group rg-saat-l60 --name vm-boot -o none

2Break it: the disk is gone, the VM won’t boot

The data disk is detached to move it to another VM, and the VM is restarted. Wait 3 minutes after the restart, then look:

az vm disk detach --resource-group rg-saat-l60 --vm-name vm-boot --name data1 -o none
az vm restart --resource-group rg-saat-l60 --name vm-boot -o none
az vm get-instance-view --resource-group rg-saat-l60 --name vm-boot --query "instanceView.vmAgent.statuses[0].displayStatus" -o tsv
az vm boot-diagnostics get-boot-log --resource-group rg-saat-l60 --name vm-boot | Select-String -Pattern 'emergency|Dependency failed|Timed out|root account' | Select-Object -Last 4

You should see something like this:

Not Ready
[ TIME ] Timed out waiting for device dev-disk-by\x2duuid-3f1c2a9e....device - /dev/disk/by-uuid/3f1c2a9e-5b7d-4c11-9e2f-8a6d4b0c7e15.
[DEPEND] Dependency failed for data.mount - /data.
You are in emergency mode. After logging in, type "journalctl -xb" to view system logs,
Cannot open access to console, the root account is locked.

The agent is Not Ready, so Run Command won’t work. The boot log names the problem exactly: /data’s device is missing and the system is in emergency mode. And the console can’t help, because emergency mode wants the root password and root is locked on Azure Ubuntu images.

Fix it: repair the OS disk on a repair VM with Azure Linux Auto Repair (about 15 minutes in total):

az vm repair create --resource-group rg-saat-l60 --name vm-boot --repair-username repairadmin --repair-password $pw --verbose
az vm repair run --resource-group rg-saat-l60 --name vm-boot --run-id linux-alar2 --parameters fstab --run-on-repair --verbose
az vm repair restore --resource-group rg-saat-l60 --name vm-boot --verbose

Then check the VM:

az vm get-instance-view --resource-group rg-saat-l60 --name vm-boot --query "instanceView.vmAgent.statuses[0].displayStatus" -o tsv
az vm run-command invoke --resource-group rg-saat-l60 --name vm-boot --command-id RunShellScript --scripts 'grep data /etc/fstab; ls /etc/fstab*' --query "value[0].message" -o tsv

You should see something like this:

Ready
[stdout]
UUID=3f1c2a9e-5b7d-4c11-9e2f-8a6d4b0c7e15 /data ext4 defaults,nofail 0 2
/etc/fstab  /etc/fstab.copy

ALAR copied the original file and added nofail, so a missing data disk no longer blocks boot. The VM is back, the agent is Ready, and you can attach the disk again whenever you want.

3Clean up

The repair tooling keeps the original OS disk in case you need it. Delete everything with the resource group:

az group delete --name rg-saat-l60 --yes
az group list --query "[?starts_with(name, 'repair-')].name" -o tsv

The second command should print nothing; if a repair group is left over, delete it too.

✅ Checkpoint: You re-enabled boot diagnostics, read an emergency-mode boot log, and repaired an unbootable VM without SSH, Run Command or root.