Azure Monitor Agent not sending data

Keep web01 running and the NAT gateway in place for both breaks.

1Break it: heartbeat is fine, but the app’s logs are missing

The app on web01 writes an informational message to syslog, and its team says it never shows up:

az vm run-command invoke --resource-group rg-saat-vms --name web01 --command-id RunShellScript --scripts 'logger -p user.info saatvedha app started; logger -p user.warning saatvedha disk almost full; echo sent' --query "value[0].message" -o tsv
Start-Sleep -Seconds 300
az monitor log-analytics query --workspace $ws --analytics-query "Syslog | where TimeGenerated > ago(30m) and SyslogMessage has 'saatvedha' | project TimeGenerated, SeverityLevel, SyslogMessage" -o table

You should see only the warning:

TimeGenerated                 SeverityLevel    SyslogMessage
----------------------------  ---------------  ---------------------------
2026-10-21T09:47:12.0000000Z  warning          saatvedha disk almost full

The agent works; the warning arrived. The info message was never collected. Look at what the DCR asks for:

$sub = az account show --query id -o tsv
$dcrId = "/subscriptions/$sub/resourceGroups/rg-saat-core/providers/Microsoft.Insights/dataCollectionRules/dcr-saat-basic"
az rest --method get --url "https://management.azure.com${dcrId}?api-version=2022-06-01" --query "properties.dataSources.syslog[].{name:name, facilities:join(',', facilityNames), levels:join(',', logLevels)}" -o table

You should see something like this:

Name         Facilities                Levels
-----------  ------------------------  ---------------------------------------
syslog-base  auth,daemon,syslog,user   Warning,Error,Critical,Alert,Emergency

Lesson 32’s DCR collects Warning and above. That was a good choice for cost, and it’s why the info message never left the VM.

Fix it with a careful read-modify-write: save the current DCR, add a source for user Info and Notice only, and put it back:

New-Item -ItemType Directory -Force C:\azure-lab\l70 | Out-Null
az rest --method get --url "https://management.azure.com${dcrId}?api-version=2022-06-01" -o json | Set-Content C:\azure-lab\l70\dcr-original.json
$dcr = Get-Content C:\azure-lab\l70\dcr-original.json -Raw | ConvertFrom-Json
$src = [pscustomobject]@{ name = 'syslog-user-info'; streams = @('Microsoft-Syslog'); facilityNames = @('user'); logLevels = @('Info', 'Notice') }
$dcr.properties.dataSources.syslog += $src
$body = @{ location = $dcr.location; properties = @{ dataSources = $dcr.properties.dataSources; destinations = $dcr.properties.destinations; dataFlows = $dcr.properties.dataFlows } }
$body | ConvertTo-Json -Depth 20 | Set-Content C:\azure-lab\l70\dcr-new.json
az rest --method put --url "https://management.azure.com${dcrId}?api-version=2022-06-01" --body '@C:\azure-lab\l70\dcr-new.json' --query "properties.provisioningState" -o tsv

You should see Succeeded. The agent picks up the change within a few minutes. Send a new info message and query again:

az vm run-command invoke --resource-group rg-saat-vms --name web01 --command-id RunShellScript --scripts 'logger -p user.info saatvedha app restarted; echo sent' --query "value[0].message" -o tsv
Start-Sleep -Seconds 300
az monitor log-analytics query --workspace $ws --analytics-query "Syslog | where TimeGenerated > ago(10m) and SyslogMessage has 'saatvedha app' | project TimeGenerated, SeverityLevel, SyslogMessage" -o table

The info message now appears.

2Break it: the identity is removed

Someone cleans up “unused” managed identities, including web01’s:

az vm identity remove --resource-group rg-saat-vms --name web01 -o none
Start-Sleep -Seconds 900
az monitor log-analytics query --workspace $ws --analytics-query "Heartbeat | where Computer == 'web01' | summarize last_heartbeat = max(TimeGenerated)" -o table
az vm run-command invoke --resource-group rg-saat-vms --name web01 --command-id RunShellScript --scripts 'tail -n 3 /var/opt/microsoft/azuremonitoragent/log/mdsd.err' --query "value[0].message" -o tsv

The last heartbeat stops moving forward, and the agent log shows errors about getting a token or authenticating, something like this:

Last_heartbeat
----------------------------
2026-10-21T09:53:40.0000000Z
[stdout]
... Failed to get access token from IMDS ... identity not found ...

Without an identity, AMA can’t prove which VM it is, so Azure Monitor refuses its data. Nothing changed in the DCR or the network. An identity can look “unused” because no role assignment refers to it, which is exactly the AMA case.

Fix it: give web01 a system-assigned identity again and wait for the heartbeat:

az vm identity assign --resource-group rg-saat-vms --name web01 -o none
Start-Sleep -Seconds 600
az monitor log-analytics query --workspace $ws --analytics-query "Heartbeat | where Computer == 'web01' | summarize last_heartbeat = max(TimeGenerated)" -o table

The heartbeat is current again.

3Clean up

Put the DCR back the way it was, remove the NAT gateway and deallocate web01:

$orig = Get-Content C:\azure-lab\l70\dcr-original.json -Raw | ConvertFrom-Json
@{ location = $orig.location; properties = @{ dataSources = $orig.properties.dataSources; destinations = $orig.properties.destinations; dataFlows = $orig.properties.dataFlows } } | ConvertTo-Json -Depth 20 | Set-Content C:\azure-lab\l70\dcr-restore.json
az rest --method put --url "https://management.azure.com${dcrId}?api-version=2022-06-01" --body '@C:\azure-lab\l70\dcr-restore.json' -o none
az network vnet subnet update --resource-group rg-saat-core --vnet-name vnet-saat --name snet-web --remove natGateway -o none
az network nat gateway delete --resource-group rg-saat-core --name nat-l70
az network public-ip delete --resource-group rg-saat-core --name pip-nat-l70
az vm deallocate --resource-group rg-saat-vms --name web01

✅ Checkpoint: You found three different reasons for “no data”: network, DCR content and identity, and you put everything back as it was.