AI Agent + Home Assistant: A Realistic Integration

The Home Assistant Landscape in 2026

Home Assistant already ships its own conversation agents. Since the September 2025 "AI in Home Assistant" release, you can plug OpenAI, Anthropic, Google, or a local Ollama server directly into the Assist pipeline and have it control your devices, write automations, and answer questions about your home. If you followed the trail this year, the big change is that small function-calling models like Qwen3 8B now handle those tool calls reliably on a mini PC, so the whole loop can stay local.

That covers a real chunk of what people search for when they type "ai agent home assistant integration." It does not cover all of it. What Home Assistant does not give you out of the box is a persistent, message-driven agent that lives outside the house, remembers things across seasons, runs scheduled checks, and reaches you on your phone when nobody is standing in front of a wall panel. That is where an external agent, connected over MCP, earns its keep.

What the Built-in Conversation Agents Do Well

Before you add anything, know what you already have. Home Assistant's Assist pipeline is the routing layer: microphone in, action out. You attach a conversation agent to it, and that agent decides how a spoken command becomes a service call.

The current supported paths:

OpenAI, Anthropic, Google conversation integrations - cloud model, fast, good tool-calling. Your voice and text go to the vendor.
Ollama integration - points at a local Ollama server. In 2026 this is production-viable for households with a mini PC or NUC. Qwen3 8B is the current sweet spot for tool-calling latency.
Local LLM Conversation - a generic OpenAI-compatible client. Useful if you self-host something other than Ollama (llama.cpp, vLLM, LM Studio).

Any of these three is enough for the base use case: "turn off the kitchen lights," "set the thermostat to 20," "is the garage door open?" You do not need Hermes for that. If your goal is a private, in-house voice assistant that talks to your devices, pick the Ollama path and stop reading here.

Where the Built-in Agents Fall Short

The built-in agents are tightly scoped by design: they answer, they act, and then they wait for the next utterance. That model breaks down in three places.

No persistent context across time. Ask Assist "what did we spend on electricity last month" and it will look at whatever the current context lets it see. It does not remember that last month you asked to bring the standby load down, or that you swapped the fridge in April. Every session starts from scratch.
No proactive schedule. The Assist pipeline is reactive. You can build classic automations that fire on triggers, and you can write LLM-authored automations with the Suggest button, but you cannot easily ask an agent to "every Sunday morning, look at last week's energy data, tell me which devices ran longest, and flag anything that looks off." That is agent work, not automation work.
No conversational surface outside the house. Assist lives in the app and on Voice hardware. If you want to check the boiler while at the office, or ask your agent to prep a weekly summary and DM it to you, you leave the Home Assistant surface entirely.

Those gaps are not bugs. They are what happens when a conversation agent is designed to be a control layer, not a co-pilot.

The MCP Bridge That Changed the Picture

The Model Context Protocol integration Home Assistant shipped in 2025 is the missing piece. Home Assistant now runs a built-in MCP server that exposes your devices, states, areas, and Assist intents as MCP tools. Any MCP-aware agent can connect to that server and control your house with the same primitives Assist uses.

This is the bit that unblocks a real external agent. Instead of building a custom Home Assistant integration for every AI product, you point an MCP-capable agent at your Home Assistant URL, hand it a long-lived access token, and it inherits the full Assist toolset. The Home Assistant MCP server already exposes on the order of 95+ tools, covering lights, climate, security, media, sensors, and scenes.

If you have used Hermes Agent with MCP for GitHub or Slack, the mental model is identical here. The house is just another MCP server.

Where an External Agent Like Hermes Fits

An external agent does not replace Home Assistant. Home Assistant stays in charge of devices, state, and the physical hardware; that is what it is good at. The external agent sits on top and handles the pieces Assist was not designed for: memory across sessions, scheduled thinking, and a chat surface that follows you outside the walls of the house.

Concretely, a setup we recommend:

Home Assistant on a Raspberry Pi, NUC, or existing homelab. Runs the MCP server (mcp_server integration) and continues to own automations and dashboards.
Hermes Agent on a $5 VPS, on the same homelab host, or as a managed Hermify instance. Configured with the Home Assistant MCP server as one of its tool sources.
Telegram as the conversational surface. You talk to Hermes; Hermes talks to Home Assistant.

With that shape, three things become easy that were awkward before:

Persistent house memory. Hermes keeps USER.md-style memory files with facts you told it once: "the fridge is a Bosch KGN, replaced in April 2026", "we run summer setpoint at 24, winter at 20", "kids' bedtime is 20:30 on school nights." Assist forgets those between calls. Hermes remembers.
Scheduled checks with real reasoning. "Every Sunday at 10 am, summarize the past week's energy usage, tell me the top three device-hours, and flag anything above last month's baseline." Home Assistant fires the schedule, Hermes reasons over the data via MCP calls, and the result lands in Telegram.
Chat surface outside the house. You are in a meeting, and someone asks whether the office lights are still on. Ask Hermes. It queries the Home Assistant MCP server, gets state, and answers. Same Telegram thread you use for the rest of your day.

When to Pick Each Path

Choose one honestly. Adding an external agent when Assist is enough just multiplies moving parts.

Only local voice control of devices, private and fast? Ollama + Assist. Done.
Local voice control plus schedule-driven automations? Assist + regular Home Assistant automations, no external agent needed. Use the Suggest button when you write them.
Local voice control, persistent memory, external chat surface, and scheduled reasoning? Assist for the wake-word loop, Hermes over MCP for the rest. This is the setup this post is for.
No wake-word hardware, mostly chat-driven, but you still want to control devices? You can skip the wake-word side entirely and drive everything through Hermes on Telegram. Ask, get, done.

For the deeper choice between running Hermes yourself and letting Hermify manage it, we already broke that down in self-hosted vs managed Hermes Agent. Nothing about the Home Assistant integration changes that trade-off. The MCP config is the same in both cases.

A Sketch of the Wiring

The full setup is documented per component, but the shape of the config is short enough to include here so you can see what "connecting Hermes to your house" actually means.

On the Home Assistant side, enable the MCP server integration and generate a long-lived access token. You end up with an MCP endpoint you can hit and a bearer token.

On the Hermes side, add the Home Assistant MCP server to your agent configuration:

{
  "mcpServers": {
    "home-assistant": {
      "url": "https://ha.your-domain.tld/mcp_server/sse",
      "headers": {
        "Authorization": "Bearer <long-lived-access-token>"
      }
    }
  }
}

Restart the agent. It introspects the server, learns your device names, and adds them to its tool list. On the next Telegram message, "turn off the porch light" becomes a real Home Assistant service call, executed with the same permissions Assist uses.

The security surface is the standard MCP one: the token is powerful, keep it out of source control, put the Home Assistant URL behind a reverse proxy or Tailscale rather than exposing it to the open internet. If you want an end-to-end version of the private-network path, our private self-hosted assistant walkthrough covers that pattern for Hermes generally.

What This Doesn't Do

The honest limits, before you commit:

It does not replace Home Assistant. No dashboards, no ZigBee stack, no built-in blueprints. All of that stays.
It is not a real-time voice pipeline in the room. Ambient wake-word listening still belongs to Assist and your voice hardware. Hermes is chat-first with optional voice on Telegram, not a wall-panel replacement.
It does not remove the cloud entirely if you use a hosted LLM for Hermes. The MCP calls stay inside your network, but if you point Hermes at OpenRouter or a similar model provider, the reasoning happens off-box. To keep everything local, run Hermes with an Ollama-backed model provider and it becomes a fully local stack.

Where to Go Next

If you want to try the setup end-to-end and skip the server maintenance, get started with Hermify. You get a managed Hermes Agent on Telegram, connect the Home Assistant MCP server to it, and the rest is a conversation. If you would rather run the whole stack yourself, the same MCP wiring works locally; the only decision is whether you also want Hermify to manage the VPS side for you.

Either way, the split is the useful one: Home Assistant remains the source of truth for your house, and the AI agent lives one layer up, where memory and schedule and conversation actually belong.