Florian Horner Blog

Right bug, wrong fix: How my own AI report made me nuke 6 live dashboards at 3 a.m.

I run Home Assistant for our whole apartment: lighting, presence, climate, an alarm, a wall-mounted kiosk. For weeks the box had been logging websocket overload and Supervisor timeouts:

Symptoms, in log form:

Client unable to keep up with pending messages. Reached 4096 pending messages.
No ACK from MQTT server in 10 seconds (mid: 55, 56, 75, 383, 384)
Timeout connecting to Supervisor: core_mosquitto, zigbee2mqtt, matter_server, ring_mqtt
Cannot register panel at device-system-utilization, it is already defined.

So I did what everyone does now. I pasted the logs into an AI deep-research tool and asked what was wrong.

The report

What came back looked like a forensic report. Three ranked culprits, a staged rollout plan ordered by risk, impact estimates down to percentages. Excerpts, verbatim:

What the AI thought was happening:

Top 3 contributors (inferred from log signatures) [...] reconnect failures for m5-atom-echo-1 (192.168.178.84) and ble-proxy-b83c20 (192.168.178.151) [...]

Remove Sonos hosts from configuration.yaml — HA explicitly says discovery works and they're not needed.

If Zigbee2MQTT is logging at debug or verbose, drop to info; fewer logs mean less I/O [...]

Set expand_light_groups: false on every Adaptive Lighting switch [...] Modeled reduction in AL-attributable event volume: ~40–60% [...] a meaningful whole-system reduction (~10–25%).

It named my devices, my rooms, my add-ons. It read like it had been inside the machine. The fine print, though, was carrying more weight than the tables:

the numbers below are modeled estimates, not measured [...] Treat percentages as order-of-magnitude planning figures.

The audit

I didn't apply any of it. I had coding agents verify it claim by claim in two read-only lanes: one grepping the git repo that backs my whole config, one reading the live instance over Tailscale. The scorecard:

One in five checkable claims was true. The skeleton was accurate, which is exactly what makes these reports dangerous: it knew my integrations, my flaky devices, my lighting stack. When the topology checks out, you extend trust to the numbers. The numbers were where it lied.

The fix that fixed too much

So we acted on the one confirmed bug. Six stale registry records, no content files behind them, deleted through HA's own WebSocket API, the same call the dashboard UI makes. Pre-flight said nothing could go wrong: records inert, YAML dashboards canonical, nothing to lose at either end.

The “harmless” delete calls:

delete device_system_utilization: {"code": "home_assistant_error", "message": "Not supported"}
delete dashboard_multimedia:      {"code": "home_assistant_error", "message": "Not supported"}
delete smart_home:                {"code": "home_assistant_error", "message": "Not supported"}
delete dashboard_presence:        {"code": "home_assistant_error", "message": "Not supported"}
delete magic_areas:               {"code": "home_assistant_error", "message": "Not supported"}
delete mein_zuhause:              {"code": "home_assistant_error", "message": "Not supported"}

Six errors that looked like six no-ops. The core log tells it differently:

2026-07-03 03:06:39.460 ERROR (MainThread)
[homeassistant.components.websocket_api.http.connection]
Error handling message: Not supported (home_assistant_error)
<admin> from 127.0.0.1

HA's removal handler unregisters the panel at that URL first, then discovers the object occupying the URL is a YAML dashboard it cannot delete, and throws. By the time the error reaches you, the registry purge has succeeded and the live panel is gone, both at once. The verification I ran seconds later:

PANELS:
  device-system-utilization: MISSING
  dashboard-multimedia: MISSING
  smart-home: MISSING
  dashboard-presence: MISSING
  magic-areas: MISSING
  mein-zuhause: MISSING

Six dashboards vanished from the sidebar. It was 3 a.m., and I noticed on the hallway kiosk.

Recovery, with receipts

The YAML files and configuration.yaml were untouched throughout, so one restart re-registered everything:

RESTORED: all 6 panels registered
storage registry entries: ['test_lighteneer', 'dashboard_asd']
$ ha core logs | grep -ci "Cannot register panel"
0

That zero is the punchline. The records really were purged, so the box booted clean for the first time in months. Right destination, wrong route.

The aftershock

One more thing broke: CI.

The CI aftershock:

FAIL: Sidebar visibility mismatch should fail but validator returned 0
=== Results: 10 passed, 1 failed, 11 total ===

A sandbox test validates my docs-vs-config drift checker by flipping the string No (hidden) and asserting the checker notices. That string existed on exactly one dashboard row, the hidden one we had just removed. The mutation no-oped, the sandbox stayed pristine, and the test declared the checker broken when it wasn't. The fix was one line, scoping the mutation to a row that will outlive any cleanup:

-sed 's/No (hidden)/Yes/' "$SB/DESIGN.md"
+sed '/`mein-zuhause`/ s/| Yes |/| No (hidden) |/' "$SB/DESIGN.md"

What I'm keeping

Verify per claim, never per report. This one went one for five on checkable claims and still pointed at a real bug. Both facts matter.

A verified bug is not a safe fix. My pre-flight checked the data. The failure lived in the code path, an ordering bug inside a delete handler that no data check could see.

An error response is not a no-op. "Not supported" arrived after the half-execution, a receipt rather than a refusal.

Every removal finds the test that assumed presence.

Keep the hard gates. My agents are blocked from restarting Home Assistant without my explicit say-so, no exceptions. At 3 a.m., with six dashboards down, the recovery step stayed a human decision. That gate was the only thing that night which worked exactly as designed.

#agent-failures #home-assistant