Run an AI Agent on Raspberry Pi (2026): Honest Guide

The question is not whether it runs, it is whether it stays up

If you are here, you almost certainly have a Raspberry Pi sitting on a shelf and a plan to point it at a personal AI agent. The good news is that a Pi 5 with 4 GB or more of RAM will run an API-backed agent runtime like Hermes Agent, OpenClaw, or a small Ollama loop without complaint. The bad news is that most Pi-hosted agents die from something unrelated to CPU: SD-card corruption at 3 AM, a memory-pressure OOM kill in the middle of a voice transcription, or a docker image that never had an ARM64 build in the first place.

This guide walks the three decisions that actually matter for a 24/7 agent on a Pi: which board and RAM tier to pick, how to keep the storage alive, and what to do about model inference. It also names the honest conclusion nobody selling Pi accessories will: for a lot of readers, a €5/month VPS ends up cheaper and calmer than the Pi they already own.

Pi 4 vs Pi 5: the CPU jump is real, but RAM decides

The Pi 5 upgraded from the Cortex-A72 in the Pi 4 to the Cortex-A76, at 2.4 GHz vs 1.5 GHz, with larger caches and materially better instruction-level parallelism. For an agent that mostly waits on network I/O to a remote model, the CPU difference shows up in one place: how fast the messaging gateway, the tool loop, and any local audio work respond after a burst of activity. Cold start on the Pi 5 is noticeably snappier than the Pi 4 for anything Docker or Node-based.

RAM, not clock speed, is what forces the buying decision. Realistic tiers for a 2026 agent stack that talks to a hosted LLM:

• Pi 4, 2 GB: too tight. The runtime plus the messaging gateway plus a Chromium tool call will crash under memory pressure.
• Pi 4, 4 GB: works for a single agent talking to a remote model, no local Whisper, no browser tool. Expect to disable swap and be careful with concurrent processes.
• Pi 5, 4 GB: the sensible floor. Comfortable for a Hermes Agent style runtime with 1-2 messaging channels and persistent memory. Local speech-to-text via faster-whisper is possible but painful.
• Pi 5, 8 GB: the practical sweet spot for anyone who wants breathing room, an occasional headless browser tool, or a small local model. Community benchmarks for local inference back this up: the Pi 5 8 GB handles TinyLlama 1.1B at around 18 tokens/sec and Llama 3.2 3B at around 9 tokens/sec on CPU alone.
• Pi 5, 16 GB: only if you plan to run 7 B class local models. A Q4_K_M quantised 7 B fits in roughly 5 GB of RAM, so you want headroom.

Whichever tier you pick, do not skip an active cooler on the Pi 5. Sustained thermal throttling turns a snappy 2.4 GHz A76 into something worse than a Pi 4 within minutes of a real workload.

ARM64 image availability: check before you commit

Almost every serious agent runtime ships an ARM64 image in 2026, but there are still traps. Before you commit, check three things on the runtime you plan to use:

• The container image manifest lists linux/arm64/v8. Run docker manifest inspect <image> and look for the ARM entry, or use a multi-arch aware pull.
• The image was actually built recently. Some projects publish an ARM tag once and forget it, then quietly diverge from the AMD64 build for months.
• Any optional dependencies you rely on (ffmpeg, sox, Chromium, Playwright) work on ARM. Playwright's bundled browser download, in particular, does not always resolve on ARM Linux and you end up installing Chromium via apt instead.

If you are running Hermes Agent, the official Docker image is multi-arch and works on ARM. For a full walkthrough of the Docker path that ships an ARM64 build, our Docker deployment guide covers the image layout, and the container-keeps-restarting troubleshooting post is worth reading before the first 24-hour test because most Pi failures land in the same OOM-kill and volume-permission buckets.

The SD card is the part that will kill your uptime

The Pi runs an operating system that logs everything, and an AI agent adds a memory directory that writes constantly: conversation state, per-user memory files, transcripts, scheduled task history. That constant small-block write pattern is exactly what wears out a consumer microSD card. In continuous 24/7 operation, the question is not if the card fails, it is when.

Three mitigations, in order of impact:

1. Boot from NVMe or USB SSD, not the SD card. The Pi 5 supports NVMe via a HAT, and the Pi 4 boots from USB SSD out of the box on recent firmware. Random write performance is an order of magnitude better and the wear characteristics are dramatically friendlier. This is the single biggest reliability upgrade you can make.
2. If you must use SD, buy a high-endurance card. SanDisk MAX Endurance, Samsung PRO Endurance, and Transcend High Endurance are specifically rated for continuous writes. A standard consumer card in a write-heavy setup can degrade in months.
3. Move the memory directory and any SQLite databases off the boot medium. Attach a small external SSD over USB and mount it at the runtime's data path. Enable Log2Ram so systemd journals live in RAM and only flush to disk periodically. Configure tmpfs for /tmp and /var/tmp.

Regardless of which storage you land on, keep a scripted backup of the memory directory to something off-Pi. A dead SD card in a drawer is one thing; a dead SD card that held six months of your agent's context is another.

Local model or remote model: pick one before you buy

The single biggest fork in the road is whether the model runs on the Pi or somewhere else.

Remote model (recommended for most personal use): the Pi runs the runtime, the messaging gateway, the memory files, and the tool loop. Model calls go to OpenAI, Anthropic, or OpenRouter over HTTPS. Total RAM footprint is comfortably under 1 GB. This is what a €5/month VPS also does, so if you take this path the Pi's only advantage over a VPS is that it lives at your house.

Local model on the Pi: the whole thing runs offline. You are limited to small models (roughly 1 B to 3 B parameters at conversational speed on a Pi 5 8 GB) with quality that lags a modern hosted model by a wide margin. The Pi 5 with 16 GB and an NVMe boot can push a quantised 7 B into service, but you will feel every second of the higher latency. For a hobbyist or a privacy-first setup with modest expectations, this is fine. For a working assistant that actually saves time, it usually is not.

For voice, the same fork applies: cloud STT (Deepgram, Whisper API) latency will beat local faster-whisper on a Pi 5 by seconds per turn. If low-latency voice matters, the Pi is the wrong shape of hardware regardless of RAM.

The honest cost comparison

If you already own the Pi, the electricity to run a Pi 5 24/7 is roughly 3 W at idle, which comes out to about $7 to $8 per year at €0.30/kWh. That is cheap. But the total cost equation for an agent host is Pi + high-endurance SD (or NVMe HAT + NVMe SSD) + PSU + case + active cooler, and once you add those up you are often within a few euros of a year of low-end VPS hosting.

Concretely:

The Pi breaks even against a €5/month VPS at roughly year three, and only if nothing on the Pi ever fails. A single SD-card replacement or a PSU swap resets the clock. The cheap VPS comparison walks the exact specs of the €5/month options, and the Hermes cost breakdown puts the model bill in context.

The Pi wins on three axes that a VPS cannot match: it is physically at your house, it draws almost no power, and it is genuinely private in the sense that no cloud provider is holding the disk. Those are real reasons. "Cheaper" is usually not.

When the Pi is the right call

• You already own it and it is otherwise idle.
• You want the agent physically on your home network for LAN-only tools (Home Assistant, local file access, Zigbee coordinators).
• You are optimising for privacy over performance and you accept the local-model quality gap.
• You enjoy the setup as a project. That is a valid reason and this guide is not going to talk you out of it.

When to skip the Pi entirely

• You do not have the Pi yet and are pricing this from zero.
• You want low-latency voice.
• You want to run a serious hosted-quality model (any 7 B and above at real speed).
• You do not want to think about SD-card wear, thermal throttling, or ARM image availability at all.

For that last case, managed hosting exists precisely because the Sunday afternoon of dimensioning, hardening, and babysitting is not fun for everyone. Get started with Hermify and skip the hardware decision entirely. It runs a managed Hermes Agent on Telegram, with persistent memory that stays yours, and there is no board to cool or SD card to replace.

Recommendation

If you already have a Pi 5 with 4 GB or more and an NVMe HAT, put an API-backed agent on it and enjoy the low power bill. If you are pricing this from scratch, buy a €5/month ARM VPS instead. If you want the agent to just work with no hardware conversation, use a managed host.

The Pi is a lovely computer. It is not, for most people, the cheapest or the calmest way to keep an AI agent online.

Sources

• Raspberry Pi 5 LLM Benchmarks (2026): 12 Models, Real Tokens/Sec - Local AI Master
• Running LLMs on Raspberry Pi 5: A Practical Guide with Real Benchmarks - TinyWeights.dev
• Raspberry Pi Power Consumption 2026: All Models Compared - raspberry.tips
• SD Card Lifespan Calculator - raspberry.tips
• Netcup vs Hetzner After the RAMpocalypse - netcupvoucher.com