---
title: "Where Should I Run My Homelab AI Service?"
date: "2026-05-15"
publish_date: "2026-08-27"
excerpt: "I built a conversation memory system for Claude Code and immediately ate 6.5 GB of my MacBook's RAM. The obvious answer was Aurora Serverless v2. The right answer was a dusty HP MicroServer."
tags: ["aws", "homelab", "postgres", "pgvector", "ollama", "mcp", "claude-code"]
author: "Martin Harris"
canonical: "https://blog.nakomis.com/2026-05-15-where-to-run-my-homelab-ai-service"
approved: true
---

# Where Should I Run My Homelab AI Service?

![A split scene: on the left a small home server glowing on a wooden shelf in a dim study, on the right a vast cloud data centre bathed in cold blue light, conceptual, cinematic, shallow depth of field](images/2026-05-15-where-to-run-my-homelab-ai-service-1.png)

I built a system that records every Claude Code conversation to PostgreSQL, generates a vector embedding for each message using Ollama, and exposes the whole thing as an MCP (Model Context Protocol) server so Claude can search its own history. The idea is that when a session gets compacted and the context is lost, there's a searchable archive of every exchange going back months.

It works well. The problem is that running PostgreSQL in Docker plus an Ollama model loaded in memory costs about 6.5 GB of RAM — roughly split between the PostgreSQL container, the `mxbai-embed-large` model (~1.2 GB), and associated overhead. My MacBook has 24 GB. That sounds fine until you add a browser with fifteen tabs, a JVM or two, and the occasional Xcode build. The 6.5 GB starts to feel like a tax.

The obvious fix: run it somewhere else. I evaluated Aurora Serverless, RDS, Hetzner, and existing hardware.

## The options

When you write a lot of AWS infrastructure, the instinct is to reach for a managed service. Aurora Serverless v2 looked attractive. The numbers come from actual usage data — 19 days of sessions, query by query — so this isn't back-of-envelope.

> **Pricing note:** AWS figures are for us-east-1, Aurora PostgreSQL, as of mid-2026; they vary by region. Hetzner prices exclude VAT.

| Option | Monthly cost | Notes |
|--------|-------------|-------|
| Aurora Serverless v2 (0.5 ACU minimum, 24/7) | ~$43 realistic | 0.5 ACU floor means you pay even when idle; see note below |
| RDS PostgreSQL db.t3.micro | ~$15–20 | Always-on; worse for bursty usage |
| Hetzner CX22 | ~€4.35 | 4 GB RAM; Terraform provider; runs PostgreSQL + Ollama (CPU-only) |
| Old Linux box | Free | Existing hardware; upgradeable |

The usage pattern: 49 active sessions over 19 days, roughly 3.5 hours active per day, about 12,000 messages a month. Bursty, not sustained.

**A note on the Aurora estimate:** Aurora Serverless v2 does not scale to zero. At ~$0.12/ACU-hour in us-east-1, a 0.5 ACU minimum running 24/7 costs ~$43/month even if you never query it. A figure of ~$6.65 sometimes appears in ACU breakdowns — that reflects only the hours the cluster was actively processing queries during ~3.5 active hours per day; it is not a realistic monthly bill. The Aurora cost breakdown also assumes switching embeddings to Amazon Bedrock (~$0.23/month) rather than a local Ollama model; a direct comparison with the other options would use local embeddings and skip that line. Storage and I/O charges add modestly at this data volume (~120 MB, I/O well under $1/month) but are worth factoring in at scale. For a bursty personal workload, Aurora Serverless v2 is a poor fit unless you can tolerate the idle floor.

## The hidden catch

Aurora lives in a VPC. The MCP server runs on the Mac. It connects to PostgreSQL directly — `psycopg.connect(DB_URL)` — and expects to reach the database over a plain TCP connection. Getting from a MacBook to a PostgreSQL instance inside a VPC means choosing one of the following:

- **AWS Client VPN endpoint** — around $0.10/endpoint-hour in us-east-1 (≈$72/month for 24/7) plus ~$0.05/client-connection-hour. Immediately kills the economics of Aurora Serverless.
- **EC2 bastion + SSM Session Manager port forwarding** (e.g. a t4g.nano) — cheaper, no inbound security group rules, but something to provision and maintain.
- **WireGuard or Tailscale tunnel via an EC2 instance in the VPC** — avoids the managed Client VPN fee, more operationally involved.
- **RDS Data API** — if your Aurora PostgreSQL engine version supports it, this removes the need for direct VPC access entirely. It requires rewriting the client to use HTTPS rather than a native driver, and trades connection overhead for higher per-request latency and some SQL feature constraints. Worth evaluating if you're already in the Aurora ecosystem.
- **Lambda proxy** — the MCP server calls a Lambda that runs the query and returns results. Doable, but a bespoke database proxy for a personal project, with the operational cost yours to carry.

For a solo developer, the simplest low-ops path is SSM Session Manager port-forwarding via a small bastion: no inbound security group rules, minimal surface area, and you only pay for the instance when you need connectivity. Tailscale subnet routing is a close second if you prefer a persistent tunnel.

RDS PostgreSQL endpoints can technically be made publicly accessible with tight security groups, TLS, and IAM database authentication — but for a personal database that's not a security posture I wanted to adopt.

A plain RDS PostgreSQL db.t3.micro is always-on at around $15–20/month depending on region and storage, reachable via the same VPC access solutions, without any of the scale-to-near-zero benefit.

Hetzner is genuinely interesting. There's a Terraform provider, so you get IaaS without the AWS overhead. €4.35/month for a CX22 with 4 GB RAM is enough to run PostgreSQL and Ollama — but 4 GB is tight: `mxbai-embed-large` alone occupies roughly 1.2 GB in memory, PostgreSQL shared buffers add another 512 MB–1 GB, and the OS claims 0.5–1 GB, leaving limited headroom before you're swapping. The next tier up would give more breathing room. You've also traded "old hardware in the spare room" for "small VPS you pay monthly and still have to maintain."

![A compact home microserver tucked on a bookshelf at night, status LEDs glowing amber, warm domestic lighting, shallow depth of field, cinematic](images/2026-05-15-where-to-run-my-homelab-ai-service-2.png)

## The boring answer

I have a spare room. In it there is a server called Luke — an HP MicroServer N40L with ECC RAM, running Ubuntu 24.04, previously gathering dust between occasional experiments.

Key specs for this workload:

- **RAM**: Luke has 8 GB of DDR3 in upgradeable slots, which is comfortable for PostgreSQL plus Ollama with `mxbai-embed-large` loaded. Unlike the MacBook's soldered memory, the N40L's slots can take more — relevant for a workload whose footprint will grow as sessions accumulate.
- **Disk**: a spinning HDD is fine — the conversation database was 118 MB after months of sessions; it's not growing fast
- **CPU**: anything from the last decade handles CPU-only embedding generation adequately for `mxbai-embed-large`; via Ollama it runs at ~6 seconds per embedding on the N40L, which is fast enough for a hook that fires at session end
- **Power**: the N40L idles at roughly 20–40W — at $0.12/kWh, a 30W average works out to about 21.6 kWh/month, or around $2.60/month — comparable to a cheap VPS, but on hardware already paid for

The irony of a £2,000 laptop with soldered memory being the wrong tool for this job, whilst a decade-old tower with free slots is the right one, is not lost.

## What got built

The architecture has two modes. The original single-machine setup:

```
Hook fires (UserPromptSubmit / Stop)
    │
    ▼
hook/record.py
    ├─ reads ~/.claude/projects/.../conversations/*.jsonl
    ├─ inserts new messages into PostgreSQL
    └─ generates + stores vector embedding via Ollama
                    │
                    ▼
          PostgreSQL + pgvector
                    │
                    ▼
       MCP server (mcp/server.py)
              ┌─────────┴──────────┐
              ▼                    ▼
       search_memory        get_conversation
    (semantic search)     (full transcript)
```

*Figure 1: Local single-machine flow*

The client-server mode, for offloading to Luke:

```
Hook fires
    │
    ▼
Hook enqueues to ActiveMQ (STOMP)
    │
    ▼
TypeScript worker (on Luke)
    ├─ dequeues message batch
    ├─ calls Ollama for embedding
    └─ writes to PostgreSQL
```

*Figure 2: Remote queue-based flow*

The server directory has a `docker-compose.yml` that runs PostgreSQL (pgvector/pgvector:pg17), ActiveMQ, and a backup container that syncs to S3. There's a CDK stack for the backup bucket and a retention Lambda. The TypeScript worker connects to STOMP, processes message payloads, and handles reconnection with exponential backoff.

ActiveMQ was already present in the homelab stack, which is why it appears here rather than a lighter alternative like Redis or Mosquitto. For a fresh setup the lighter options would be worth evaluating; the key point is that the hook enqueues asynchronously rather than writing synchronously.

A single environment variable switches modes:

```bash
CLAUDE_CHATS_MODE=remote
```

In remote mode the hook sends messages to the queue rather than writing directly to the database. The MCP server reads from whichever `CLAUDE_CHATS_DB_URL` it's pointed at — on the Mac, that's `postgresql://claude:claude@luke.local:5433/claude_chats`. Luke is resolved via mDNS on the local network.

## Validating Luke before committing

Before migrating anything permanently, I checked the actual query latency. A search does two things: embed a query, then run a cosine similarity search. Embedding on Luke's N40L:

```bash
time curl -s http://luke.local:11434/api/embeddings \
  -d '{"model":"mxbai-embed-large","prompt":"cats Boots Chi Tau Kappa Mu"}' \
  | python3 -c "import json,sys; e=json.load(sys.stdin)['embedding']; print(f'{len(e)} dims')"
```

About 6 seconds warm, returning 1024-dimensional vectors. The PostgreSQL query itself is essentially instant at that data volume. The 6-second cost is paid on the *write* path (handled asynchronously via the queue) and on *search* (which fires manually, not on every keypress). When Claude uses the search tool it spins for a few seconds — which feels perfectly natural for a deep archive lookup.

After syncing the local database to Luke with `pg_dump` / `pg_restore` — 51 conversations, 8,226 messages at the time — a test semantic search for "cats" returned identical results via `luke.local:5433` as it had via localhost. The data was all there.

## What I actually learned

**The VPC problem is often the deciding factor.** Aurora Serverless v2 at ~$43/month idle floor sounds manageable until you price in the networking to reach it from a Mac. The managed service premium extends beyond the service cost to the connectivity required to use it.

**Old hardware is upgradeable in ways that new hardware isn't.** The MacBook's RAM is soldered. Luke's isn't. For a workload whose memory footprint will grow as more sessions accumulate, that matters more than clock speed.

**The queue architecture is worth it even on local hardware.** When the hook enqueues rather than writes synchronously, a slow embedding call doesn't hold up the Claude response. The worker processes the backlog asynchronously regardless of where it runs.

**6.5 GB back on the Mac is noticeable.** This is the part that doesn't make it into the cost model: having the RAM freed up is a qualitative change, not just a number on a spreadsheet.

---

*Martin Harris writes about connecting real hardware to abstract cloud infrastructure. If you've ever soldered something to an API Gateway, you're in the right place.*

{{donate}}