Giving Claude a Persistent Memory#

There is a character in a J.M. Barrie play — a butler named Crichton, entirely unremarkable in the drawing room — who becomes, almost quietly, the most indispensable person on the island after the shipwreck. He was always the most competent one. Normal social order just made it impossible to notice.

I think about that when context compaction kicks in mid-session and I watch several hours of careful problem-solving disappear into a summary paragraph. The work is gone. The reasoning is gone. What remains is a précis written by a version of the assistant that no longer has access to what actually happened.

So I built a memory system. This is how it works.

The Problem With Compaction#

Claude Code has a finite context window. When a session runs long, the harness compacts earlier messages into a summary and continues. The summary is accurate enough for most purposes. But it loses the texture: the specific error message that unlocked a debugging session, the particular wording of a constraint that shaped an architectural decision, the thing we tried first that didn't work and why.

There is no API to retrieve what was compacted from within the session. The only countermeasure is to record everything before it disappears.

The Stop Hook#

Claude Code exposes a hooks system. You can register commands that fire on various lifecycle events — one of which is Stop, which fires after every response. (Hooks are configured in ~/.claude/settings.json, available in Claude Code via the Claude desktop tooling.) The hook receives a JSON payload on stdin containing, amongst other things, the path to the current session transcript.

{
  "session_id": "3f047a9d-bdf4-430f-b27f-ec4d68998899",
  "transcript_path": "~/.claude/projects/.../transcript.jsonl",
  "cwd": "/Users/nakomis/repos/nakomis/claude-chats",
  "git_branch": "main"
}

The transcript is a newline-delimited JSON file. Every message is a line — each with a UUID, role, and content field. The hook reads it, finds messages not yet stored (deduped by message_uuid), embeds them, and persists them to PostgreSQL. It always exits 0 — a non-zero exit would block the stop flow. Exceptions are caught, logged to stderr, and swallowed so Claude is never impacted by hook failures.

The hook fires on UserPromptSubmit as well as Stop, which means the database stays roughly current even mid-session. Incrementality is cheap: the hook fetches already-stored message UUIDs for the session and skips them.

A separate bulk-import script handles historical sessions and backfilling — the incremental hook only covers new messages from the point of installation onwards.

Registration goes in ~/.claude/settings.json:

{
  "hooks": {
    "Stop": [{"type": "command", "command": "uv run --project \"/path/to/hook\" record-conversation", "timeout": 120}],
    "UserPromptSubmit": [{"type": "command", "command": "uv run --project \"/path/to/hook\" record-conversation", "timeout": 120}]
  }
}

The Schema#

PostgreSQL with the pgvector extension. Two tables:

-- Requires:
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pgcrypto;

CREATE TABLE conversations (
    id          UUID        PRIMARY KEY DEFAULT gen_random_uuid(),
    session_id  TEXT        NOT NULL UNIQUE,
    project_path TEXT       NOT NULL DEFAULT '',
    git_branch  TEXT        NOT NULL DEFAULT '',
    name        TEXT,
    started_at  TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE messages (
    id              UUID        PRIMARY KEY DEFAULT gen_random_uuid(),
    conversation_id UUID        NOT NULL REFERENCES conversations (id) ON DELETE CASCADE,
    message_uuid    TEXT        NOT NULL UNIQUE,
    role            TEXT        NOT NULL CHECK (role IN ('user', 'assistant')),
    content         TEXT        NOT NULL,
    content_tsv     TSVECTOR    GENERATED ALWAYS AS (to_tsvector('english', content)) STORED,
    embedding       vector(1024),
    created_at      TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    sequence_num    INTEGER     NOT NULL,
    UNIQUE (conversation_id, sequence_num)
);

The embedding column is a 1024-dimensional vector — this matches the Ollama and Bedrock Titan v2 models used in this setup. If you use a different model (nomic-embed-text outputs 768 dimensions; Cohere embed-v4 outputs 1536), adjust the column definition and index accordingly. Changing dimensions after the fact requires a migration and full re-embedding. Nearest-neighbour search uses an ivfflat index with cosine distance:

CREATE INDEX messages_embedding_idx
    ON messages USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);

CREATE INDEX messages_conversation_seq_idx
    ON messages (conversation_id, sequence_num);

A note on the lists parameter: 100 is reasonable for a corpus in the tens of thousands. Tune it as the corpus grows, and run ANALYZE messages after bulk inserts to keep the index statistics current.

There is also a full-text search index backed by the generated tsvector column above:

CREATE INDEX messages_content_tsv_idx
    ON messages USING GIN (content_tsv);

The two search paths combine via Reciprocal Rank Fusion in the hybrid mode, which is what search_memory uses by default.

The database runs in a Docker container on port 5433. Nothing exotic — pgvector/pgvector:pg17, started with a docker-compose file, init.sql mounted at the standard entrypoint path. The whole thing runs locally on my laptop, except when I'm at a client site, in which case the vectors come from Bedrock and the database is wherever I decide to put it. If the database is remote, use TLS and strict network ACLs — and keep in mind that a purge mechanism (delete by session_id) is useful if you ever need to redact a conversation. Embeddings can leak information about their source text even without the original text being stored in plain form.

The Embedding Provider Problem#

This is where the interesting decisions live.

Embeddings are expensive to change. They are baked into the database at write time. If you switch provider, the new query vectors come from a different embedding space and will not match the stored ones — cosine similarity will return nonsense, not because the content is irrelevant but because you are comparing measurements made in entirely different coordinate systems. Every stored message would need to be re-embedded from scratch.

So the provider choice is consequential.

The code supports three:

PROVIDER   = os.environ.get("CLAUDE_CHATS_PROVIDER",   "ollama").lower()
_DEFAULT_MODELS: dict[str, str] = {
    "ollama":  "mxbai-embed-large",
    "bedrock": "amazon.titan-embed-text-v2:0",
    "openai":  "text-embedding-3-small",
}

Ollama / mxbai-embed-large is the local option. The model is 670 MB, runs on-device, no API calls, no usage costs. On a modern laptop with AVX2, a single embedding takes 20–50 ms. Entirely private: the text never leaves the machine. This is what I use at home.

OpenAI text-embedding-3-small is the work option. By default this model produces 1536-dimensional vectors, but the OpenAI API supports a dimensions parameter; the embed.py in this project requests dimensions=1024 to match the schema. Faster, good quality, and the text goes to OpenAI — which is where the corporate privacy agreement matters. If your organisation has a data processing agreement with OpenAI, this is fine. If it doesn't, it isn't.

AWS Bedrock (Titan v2) is the third path. Your AWS credentials, your AWS account, your data. Useful as a middle ground: cloud-hosted compute but within an environment you control. Also useful when the machine running the hook is too slow for local inference (more on this in a moment).

The Bakeoff#

Before committing to a model, I ran a proper evaluation. Seven models — four Ollama, one Bedrock Titan, two Cohere on Bedrock — over the full message corpus (about 29,000 messages). The metric was recall@10 on a hand-labelled query set of around 50 queries: conversations I remembered having, phrased as questions, with the relevant session IDs marked as ground truth. Models with different output dimensions were evaluated against separate per-model corpora, each table created fresh with the matching vector(N) column — no stuffing 1536-d vectors into a 1024 column. Messages were embedded as raw content, one message per vector. Embedding ran on an M-series MacBook with the Ollama models, and via API for the Bedrock models.

The results, ranked by recall@10:

Model	Recall@10	Notes
amazon.titan-embed-text-v2:0	0.542	Best retrieval quality
cohere.embed-v4:0	0.509	1536-dim, throttle-prone
bge-large	0.500	670 MB, 1024-dim
cohere.embed-english-v3	0.498	Lower coverage (skips)
bge-m3	0.491	Same size as bge-large
mxbai-embed-large	0.486	Default Ollama model
nomic-embed-text:v1.5	0.413	768-dim

Titan V2 came top by a meaningful margin. The query set was built partly using mxbai candidates (which biases against Titan), so the gap is real, not an artefact. Titan is also the cheapest of the cloud options at roughly $0.00002 per 1000 tokens — about 15 pence per month at typical session volume (May 2026 pricing; see the repo README for current assumptions).

The practical lesson: for a personal project running on a laptop, the quality difference between Titan and the Ollama models is real but not enormous. For a server environment where local inference is impractical — say, an old HP MicroServer with a 2-core AMD Turion and no AVX2 — Titan wins on both quality and latency. Six seconds per embedding on that hardware, 40 messages per session, 120-second hook timeout: the maths does not work.

The Local Model Privacy Case#

The security angle on running Ollama at work deserves its own treatment.

The instinct — and I have felt it — is to just install Ollama in a Docker container on your work laptop and not mention it. It runs locally. Nobody needs to know. You get local inference, total privacy, and you avoid the conversation.

This is the same energy as the person who installed some unapproved open-source tool on the accounts server because it made running payroll so much easier. They were not wrong about the productivity argument. They were wrong about the process.

The privacy case for local models is genuinely strong. When you send conversation text to an API — even one with a corporate data processing agreement — the text leaves your machine, crosses a network, and touches a third-party system. With Ollama, none of that happens. The model weights are on your disk, the inference is on your CPU, and the resulting vector is the only thing that goes into the database. That argument is strong enough to just make. Raise it with your security team. Ask for an approved deployment. The alternative — a shadow AI stack that the security team doesn't know about — is genuinely bad and usually ends with an incident report rather than a policy update.

If the answer is no, OpenAI or Bedrock with a signed DPA is meaningfully better than nothing. The privacy is imperfect but the provenance is documented and the risk is understood.

The MCP#

With data in the database, the retrieval side is a FastMCP server with three tools:

@mcp.tool()
def search_memory(query: str, limit: int = 5,
                  project_path: str | None = None,
                  mode: str = "hybrid") -> dict:
    ...

@mcp.tool()
def get_conversation(session_id: str,
                     start_seq: int | None = None,
                     end_seq: int | None = None) -> dict:
    ...

@mcp.tool()
def list_recent_sessions(limit: int = 10,
                         project_path: str | None = None) -> list[dict]:
    ...

search_memory supports three modes. Hybrid (default) runs both vector similarity and full-text search, then combines scores with Reciprocal Rank Fusion (k=60 for both legs; see search.py in the repo to adjust). Semantic-only uses the embedding for concept-level retrieval — phrasing the query as a sentence that would appear in the target conversation ("HyDE", Hypothetical Document Embeddings) improves results noticeably. Full-text is plain PostgreSQL tsvector — reliable for exact phrases, error messages, and anything where you remember the specific words.

When hybrid misses, full-text with a verbatim distinctive phrase is usually the fallback. They are not the same tool pointed at the same problem — they cover different failure modes: semantic framing differences versus exact-phrase recall.

get_conversation takes a session ID and returns the full transcript, with optional sequence number windowing for long conversations. list_recent_sessions does what it says.

The whole thing is registered via claude mcp add conversation-memory with the required environment variables — at minimum DATABASE_URL, CLAUDE_CHATS_PROVIDER, and either OLLAMA_MODEL or BEDROCK_REGION depending on your provider. The hook and MCP use the same embed.py — same provider, same model, same dimensionality.

The Provider Mismatch Incident#

Earlier today, semantic search was returning near-zero scores for everything. Full-text still worked; the data was clearly there. But cosine similarity was broken — the search embedding and the stored embeddings were not from the same space.

The diagnosis: meta-mcp/config.toml had CLAUDE_CHATS_PROVIDER = "ollama" for the conversation-memory MCP. The stored vectors were Bedrock Titan, embedded by the hook which had CLAUDE_CHATS_PROVIDER = "bedrock". The MCP was generating query embeddings with mxbai-embed-large and comparing them against Titan vectors. The two embedding spaces are completely different mathematical objects — a cosine similarity of effectively 0.0 is the correct answer when comparing them, just not the useful one.

Fix: update the MCP config to CLAUDE_CHATS_PROVIDER = "bedrock". Semantic search immediately restored. No re-embedding required because the stored data was fine all along.

The provider must match — between the hook that writes and the MCP that reads — or you are measuring the wrong thing.

What I Actually Learned#

A few things that did not appear obvious until I built it:

Hook timeout is the practical limit for local inference on low-core hardware. On a 2-core server with no AVX2, you will hit the 120-second ceiling before the session is fully embedded. Bedrock with parallel workers sidesteps this entirely.

HyDE works. Phrasing the search query as a sentence that would appear in the target conversation, rather than keywords, consistently produces better semantic matches. For example, querying "We ended up deciding to use a connection pool after hitting timeout errors" retrieves the relevant session more reliably than "connection pool timeout". The first is closer to how the actual dialogue is phrased.

Provider mismatch is silent and plausible-looking. The scores cluster around 0.015 or so, which looks like "no results" rather than "completely broken". You might conclude that the relevant conversation was never recorded. It was; the query was just in a different language.

The repo is at github.com/nakomis/claude-chats. install.sh does the full setup. Set CLAUDE_CHATS_PROVIDER before running.

---

Martin Harris writes about connecting real hardware to abstract cloud infrastructure. If you've ever soldered something to an API Gateway, you're in the right place.

---

If you find this blog useful, please consider a donation.