Semantic Search on a Static Blog (and Why 'RAG' Returns Nothing)

cv.nakomis.com has a chat feature. You talk to an AI that represents me — it knows my CV, my GitHub repos, my interests, and it can search my blog posts when something relevant comes up. The way the blog search originally worked was blunt: there was a get_blog_posts tool that loaded every post in full and dumped them all into the Claude context window. Three posts, fine. Ten posts, uncomfortable. Twenty posts, expensive and slow.

The right answer is RAG — Retrieval-Augmented Generation. Store the content as vectors, embed the query, retrieve only the relevant bits. I'd been meaning to build it properly for a while. When I sat down to do it, I ended up building two things: a RAG pipeline for the chat Lambda, and a search box for the blog itself.

This is the story of both.

Where do you put the vectors?

The usual options for a vector store are a purpose-built database (pgvector, Pinecone, Weaviate, and a dozen others), or a serverless approximation. I have three blog posts. A full database is operational overhead for something that fits comfortably in memory. Aurora Serverless has cold-start latency. A persistent Postgres instance costs money every day whether or not anyone searches.

So I worked backwards from the constraints. The posts are small. Chunked at paragraph level, they produce 73 chunks. Each chunk gets a 1024-dimensional embedding. At 4 bytes per float, that's roughly 300 KB of actual vector data — call it 1.6 MB with JSON overhead. That's a reasonable S3 object.

The plan: store the embeddings as a single JSON file in S3. At Lambda cold start, download the file and cache it in a module-level variable. Do cosine similarity in-process. No database, no persistent infrastructure, one cold-start S3 read that amortises across the container's lifetime.

// Module-level cache — loaded once per Lambda container
let chunksCache: BlogChunk[] | null = null;

async function loadChunks(): Promise<BlogChunk[]> {
    if (chunksCache) return chunksCache;
    const result = await s3.send(new GetObjectCommand({
        Bucket: process.env.PRIVATE_BUCKET,
        Key:    'blog-embeddings.json',
    }));
    chunksCache = JSON.parse(/* stream to string */);
    return chunksCache;
}

The first search after a cold start pays the S3 read cost (~200ms). Every subsequent search in the same container hits the in-memory cache. Given that blog searches are infrequent and Lambda containers survive for minutes to hours between invocations, this is a reasonable trade.

Chopping up the posts

The ingestion script runs locally. It downloads each post from S3, strips the YAML frontmatter, and walks through the body looking for paragraph breaks.

A few rules make the chunks more useful:

Code blocks are skipped entirely. Embedding raw code produces poor retrieval signal — the surrounding prose explains what the code does, and that's what you want to match against.
Each chunk is prefixed with the nearest section heading before embedding. The heading carries topical context that a bare paragraph wouldn't have — "The MQTT detour I should not have taken\n\nIt took about twenty minutes..." embeds very differently from the same paragraph without that context.
Paragraphs shorter than 80 characters get merged with the next. A single transitional sentence isn't worth a vector slot.

def chunk_body(body: str) -> list[tuple[str, str]]:
    chunks = []
    current_heading = ""
    current_lines = []
    in_code_block = False

    for line in body.splitlines():
        if line.startswith("```"):
            in_code_block = not in_code_block
            flush()
            continue
        if in_code_block:
            continue
        heading_match = re.match(r"^(#{1,3})\s+(.+)", line)
        if heading_match:
            flush()
            current_heading = heading_match.group(2).strip()
            continue
        if not line.strip():
            flush()
            continue
        current_lines.append(line.strip())
    # ...

The embedding model is Bedrock Titan Embed v2 (amazon.titan-embed-text-v2:0): 1024 dimensions, normalised. The same model runs at query time in the Lambda, so the query vector and document vectors live in the same space. That last point turned out to matter more than I expected.

The dimension mismatch

My first attempt at the ingestion script used a local Ollama model — nomic-embed-text — rather than Bedrock. It was faster to iterate locally, didn't need AWS credentials for every test run, and produced embeddings happily. I ran it against all three posts, uploaded the JSON to S3, and wired up the Lambda.

The Lambda used Bedrock Titan for query-time embedding. nomic-embed-text produces 768-dimensional vectors. Titan produces 1024-dimensional ones. Cosine similarity between a 768-dim query vector and a 1024-dim document vector is not just wrong — it's undefined. In practice, the code crashed rather than silently returning garbage, which was helpful.

The fix was to use Bedrock Titan for ingestion too. Re-ran the script, re-uploaded the JSON, moved on. The lesson is obvious in hindsight: the query-time and index-time embedding models must be the same, or at minimum produce the same dimensionality. In a proper vector database this is enforced by the schema. In a JSON file, it's enforced by you not making that mistake.

The retriever

The Lambda side is straightforward once the embedding model is settled. At query time, embed the query with Titan, compute cosine similarity against every stored chunk, return the top results above a threshold:

function cosineSimilarity(a: number[], b: number[]): number {
    let dot = 0, normA = 0, normB = 0;
    for (let i = 0; i < a.length; i++) {
        dot   += a[i] * b[i];
        normA += a[i] * a[i];
        normB += b[i] * b[i];
    }
    return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

Because the vectors are normalised, the dot product alone would be sufficient — but the explicit normalisation makes it correct regardless of whether the embedding API changes its normalisation behaviour. Compute both, divide, no surprises.

The retrieval threshold is set at 0.3. Below that, the match is weak enough that returning it would do more harm than good — the chat agent would confidently cite an irrelevant post. The first version had no threshold at all, which produced entertaining results: "RAG" returning a section about CloudFront caching.

There was also a deduplication bug in the first pass. With TOP_K = 4 and only three posts, the top results would sometimes contain two or three chunks from the same post. The fix keeps only the highest-scoring chunk per post before applying the limit:

const seen = new Set<string>();
const deduplicated = scored.filter(({ chunk }) => {
    if (seen.has(chunk.post_slug)) return false;
    seen.add(chunk.post_slug);
    return true;
}).slice(0, TOP_K);

The chat tool now calls search_blog with the visitor's question rather than get_blog_posts. The agent gets back the two or three most relevant excerpts instead of several thousand words of prose. Context window cost drops accordingly.

One more thing

Halfway through wiring up the chat tool, the obvious thought arrived: if I'm building vector search for the AI, the blog itself should have a search box.

The blog is a React SPA served from S3 — no backend, no server. A search box needs an endpoint. I added the search Lambda to the existing API Gateway (the same one that handles the nakom.is chat endpoint) and added a CloudFront behaviour on the blog's distribution to proxy /api/search there. The blog frontend calls /api/search — same domain, no CORS.

There's a CDK wrinkle: the blog site and nakom.is are two separate CDK applications in two separate repositories. The blog's CloudFront distribution needs to know the API Gateway domain, but there's no shared CloudFormation deployment to export it from. The solution is SSM: the nakom.is stack writes the domain (and later the API key) to SSM Parameters at deploy time; the blog stack reads them using valueFromLookup at synth time.

// In ApiGatewayStack (nakom.is): store the domain after stack creation
new ssm.StringParameter(this, 'BlogSearchApiDomainParam', {
    parameterName: '/nakom.is/blog-search-api-domain',
    stringValue: cdk.Fn.select(2, cdk.Fn.split('/', this.gateway.url)),
});

// In BlogStack (blog-app): read it at synth time
const blogSearchApiDomain = ssm.StringParameter.valueFromLookup(
    this, '/nakom.is/blog-search-api-domain',
);

valueFromLookup makes a real SSM API call during cdk synth and embeds the concrete string in the CloudFormation template. No cross-stack export/import coupling, no circular dependencies between CDK apps. The blog stack doesn't need to know anything about the nakom.is stack at CloudFormation runtime — it just has a hardcoded domain that happens to come from SSM.

The numbers

Once the search box was live, I started testing it. "MQTT Alexa" scored 0.46 and returned the right post. "Home automation" scored 0.36 and also worked — those exact words don't appear in the post, which is a good sign.

Then I tried "RAG."

Search scores (top 5):
  0.0880 the-certificate-that-had-to-live-in-america: The resulting architecture
  0.0801 alexa-flash-blue: Alexa, Flash Blue
  0.0797 alexa-flash-blue: The MQTT detour I should not have taken

0.088. The threshold is 0.3. No results.

The certificate post scored highest for a query about retrieval-augmented generation. This is surprising until you think about it: none of my posts contain the word "RAG", "embeddings", or "vector store". The RAG pipeline I just built lives in Lambda code, not prose. The search was correct; my expectation was wrong.

"Wireless microcontroller" was more instructive. I have a post about an ESP32 — a wireless microcontroller — connecting to AWS IoT Core. It scored 0.23. Below the threshold.

But worse: the highest score for "wireless microcontroller" was 0.25, on the Trello MCP server post. Titan apparently associates "MCP" with "microcontroller" closely enough to beat the actual ESP32 content. A lower threshold wouldn't just surface the right post — it would surface the wrong one first.

The gap

This is the gap that sits between "semantic search" and "concept search." Titan Embed v2 handles paraphrase well — "home automation" finds a post about voice-controlled LEDs because the post uses that vocabulary nearby. But "wireless microcontroller" and "ESP32" are domain synonyms that don't share vocabulary, and Titan doesn't reliably connect them.

Keyword search has the opposite problem: no synonyms at all. Titan sits in the middle — better than BM25, worse than a human reader. The question is whether you can close that gap without switching to a more capable (and more expensive) embedding model.

HyDE: write the answer first

The fix is called HyDE — Hypothetical Document Embeddings. The idea is simple enough that it's slightly embarrassing it took a paper to formalise it.

Instead of embedding the query directly, you ask an LLM to write a short passage that a relevant document might contain. Then you embed that instead of the raw query. The query vector stops being the embedding of three words and becomes the embedding of a plausible paragraph about the topic — which sits much closer to actual content in the vector space.

For "wireless microcontroller", Haiku generated:

When I first started working with ESP32 modules, I was amazed by how seamlessly they integrated wireless connectivity into small embedded projects...

That paragraph, embedded, scores 0.49 against the Alexa/ESP32 post. Up from 0.23. The correct result surfaces above the threshold.

The implementation is a small module that sits in front of the search Lambda:

const body = JSON.stringify({
    anthropic_version: 'bedrock-2023-05-31',
    max_tokens: 256,
    system: 'Given a search query, write a short paragraph (2-4 sentences) that ' +
            'a relevant technical blog post might contain as body text. ' +
            'Do not answer the query directly. Just write the hypothetical passage.',
    messages: [{ role: 'user', content: query }],
});

const response = await bedrock.send(new InvokeModelCommand({
    modelId: 'us.anthropic.claude-3-5-haiku-20241022-v1:0',
    body: Buffer.from(body),
    // ...
}));

Note the us. prefix on the model ID. That's not a typo.

The cross-region surprise

Newer Claude models on Bedrock require a cross-region inference profile rather than direct model invocation. The bare model ID anthropic.claude-3-5-haiku-20241022-v1:0 returns:

ValidationException: Invocation of model ID anthropic.claude-3-5-haiku-20241022-v1:0
with on-demand throughput isn't supported. Retry your request with the ID or ARN
of an inference profile that contains this model.

The profile ID us.anthropic.claude-3-5-haiku-20241022-v1:0 works. The us. prefix routes the request across US regions for capacity and resilience — Bedrock picks whichever of us-east-1, us-east-2, or us-west-2 has capacity at that moment.

Which creates an IAM problem. My Bedrock policy granted bedrock:InvokeModel on arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-haiku-.... The first invocation hit us-east-2 and was denied:

AccessDeniedException: ...is not authorized to perform: bedrock:InvokeModel
on resource: arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-3-5-haiku-...

The fix is a wildcard on the region, plus an explicit resource for the inference profile itself:

blogSearchFunction.addToRolePolicy(new iam.PolicyStatement({
    actions: ['bedrock:InvokeModel'],
    resources: [
        // Titan for embeddings — always us-east-1
        'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0',
        // Haiku via cross-region profile — can land on any US region
        'arn:aws:bedrock:*::foundation-model/anthropic.claude-3-5-haiku-20241022-v1:0',
        `arn:aws:bedrock:us-east-1:${this.account}:inference-profile/us.anthropic.claude-3-5-haiku-20241022-v1:0`,
    ],
}));

Two resources for Haiku: one for the foundation model in any region (where the cross-region profile actually executes), one for the inference profile itself (where the initial call lands). Without both, one or the other fails depending on which region Bedrock picks.

HyDE also degrades gracefully. If the Haiku call fails for any reason, the module logs a warning and falls back to the raw query. The search still works — just with Titan-level concept matching rather than HyDE-level matching. The threshold catches truly irrelevant results either way.

Planning for going viral

HyDE costs money per search. Not much — roughly $0.001 per query when you account for the Haiku call and the Bedrock embedding. At realistic traffic (a few dozen searches a month), that's about 4p.

I always try to reduce running costs to as close to zero as reasonably possible — not to be stingy, but because it forces you to understand the pricing structure properly. Working out what something costs requires knowing how the service is billed, which means understanding the units, the free tiers, the quotas, and the edge cases. That discipline transfers directly to designing systems at work, where the numbers have more zeroes on them.

In this case, understanding the cost meant I noticed something more important: predictability. A personal blog has no revenue model, no budget for surprises. If a post gets picked up somewhere and suddenly attracts 100,000 searches in a day, the Haiku calls alone would cost around $88. Not catastrophic, but not the sort of thing you want to discover on your AWS bill while eating breakfast.

The standard answer to this on AWS is API Gateway usage plans — a quota that hard-caps the number of requests an API key can make in a given period. The problem is that usage plans require an API key, and I didn't want a key sitting in the frontend JavaScript where anyone can read it.

Turns out this is a solved problem. The existing chat endpoint on nakom.is has always used an API key — it's just that CloudFront injects it as a custom origin header on the way to the API Gateway, so the browser never sees it:

const apiOrigin = new origins.RestApiOrigin(gateway, {
    customHeaders: { 'x-api-key': apiKeyString }
});

The key lives in SSM, gets read at CDK synth time, and ends up baked into the CloudFormation template for the CloudFront distribution. Visitors can inspect every request their browser makes and will never find it. I used the same pattern for blog search.

The blog search gets its own API key with a separate usage plan: 50 requests per day globally, burst of 2. That caps the absolute worst-case cost at around £1.50 a month regardless of how much traffic hits the site. If the blog does go viral, I'll just have to ration the cat treats 🐈.

const blogSearchUsagePlan = gateway.addUsagePlan('BlogSearchUsagePlan', {
    throttle: { rateLimit: 1, burstLimit: 2 },
    quota: { limit: 50, period: api.Period.DAY },
});

The API Gateway domain is stored in SSM; the blog's CloudFront distribution reads it at synth time and routes /api/search there with the key injected. Two CDK apps, no CloudFormation cross-stack dependency, no key in the browser.

When the quota is exceeded, API Gateway returns a 429. The frontend catches it:

if (response.status === 429) {
    throw new Error('Search is temporarily disabled due to global usage quotas');
}

Honest, unglamorous, and means I won't need to remortgage.

What I learned

A few things that weren't obvious going in:

S3 JSON is a perfectly reasonable vector store at small scale. The 1.6 MB cold-start read takes a few hundred milliseconds once and is then cached for the container's lifetime. A purpose-built vector database adds operational cost and latency for no benefit at this scale. The right tool for three blog posts is not the same tool as for three million documents.
The query-time and index-time embedding models must match. Not just the same model family — the same model producing the same dimensionality. In a vector database this is enforced by the schema. In a JSON file, it's enforced by you. I learned this by getting it wrong.
Titan Embed v2 sits between keyword search and true semantic search. It handles paraphrase and topical proximity well. It struggles with domain-specific synonyms where the vocabulary doesn't overlap. "ESP32" and "wireless microcontroller" are the same thing to a hardware engineer; Titan doesn't reliably connect them.
HyDE is the cleanest fix for that gap. Generating a hypothetical document is slightly absurd — you're asking an LLM to hallucinate a passage you wish existed — but the resulting vector is dramatically better anchored than a raw three-word query. "Wireless microcontroller" goes from 0.23 to 0.49. At ~100 searches a month, the Haiku cost is around 9p.
Cross-region inference profiles on Bedrock require IAM policies that cover all potential execution regions. The profile routes the request; you don't control where it lands. A policy scoped to us-east-1 will randomly deny requests that hit us-east-2. Wildcard the region for any model invoked through a cross-region profile.
Feeding post tags into the HyDE prompt closes the vocabulary gap further. Each post has tags (rag, mqtt, esp32, etc.). Passing the full tag list to Haiku in the system prompt — "the blog uses tags including: aws, bedrock, rag..." — means a search for "RAG" produces a hypothetical passage that actually uses the word, which scores well against the post that has it as a tag. Tag curation becomes part of search quality.

Martin Harris writes about connecting real hardware to abstract cloud infrastructure. If you've ever soldered something to an API Gateway, you're in the right place.

If you find this blog useful, please consider a donation.

Semantic Search on a Static Blog (and Why 'RAG' Returns Nothing)#

Where do you put the vectors?#

Chopping up the posts#

The dimension mismatch#

The retriever#

One more thing#

The numbers#

The gap#

HyDE: write the answer first#

The cross-region surprise#

Planning for going viral#

What I learned#