---
title: "Two Models Walk Into a Lambda"
date: "2026-05-15"
publish_date: "2026-09-23"
excerpt: "The BootBoots catcam needed a fast ML model for real-time deterrent decisions. It also needed something that could explain its reasoning. Running both at once turned out to be the interesting part."
tags: ["aws", "sagemaker", "lambda", "machine-learning", "anthropic", "bootboots", "cats"]
author: "Martin Harris"
canonical: "https://blog.nakomis.com/2026-05-15-parallel-inference-sagemaker-claude-vision"
approved: true
---

# Two Models Walk Into a Lambda

![Two silhouetted analysts examining the same photograph of a sleek black cat on a glowing lightbox, one holding a stopwatch, one a magnifying glass, in a dark shadowy room](images/parallel-inference-sagemaker-claude-vision-1.png)

The BootBoots catcam spots Boots — the neighbour's solid black cat with bright green eyes — and triggers a deterrent before he settles in somewhere he shouldn't be. The whole pipeline runs on an ESP32-CAM: capture a JPEG, POST it to API Gateway, Lambda handles the rest.

I budget about a second for the decision so the ESP32 doesn't sit around retrying. API Gateway can wait far longer, but the device shouldn't. That's fine for a SageMaker endpoint. It's less fine if you're also waiting on a general-purpose vision model. So I gave it both, and made sure neither one delays the other.

## The model

The SageMaker endpoint runs a MobileNetV2 classifier trained on labelled catcam images. The model isn't binary — it has seven classes: Boots, Chi, Kappa, Mu, NoCat, Tau, and Wolf. (Wolf is a stuffed toy we use as a test subject — a negative control that's easier to pose than a real cat.)

Class order is alphabetical, which matters because inference returns a probability array rather than a label. Boots is index 0. The deterrent decision is triggered if `probs[0]` crosses a configurable threshold.

```typescript
// Cat names in order corresponding to the probability array indices
// Alphabetical order (matches Keras class discovery): Boots, Chi, Kappa, Mu, NoCat, Tau, Wolf
// CAT_NAMES is also reused to validate Claude's cat field — which is why NoCat and Wolf are in the list
const CAT_NAMES = ['Boots', 'Chi', 'Kappa', 'Mu', 'NoCat', 'Tau', 'Wolf'];
```

The training script notes this explicitly: `Boots is index 0 — inference Lambda checks probs[0] for spray decision`. It's the kind of comment you write because you already got it wrong once — a retrained model came back with shifted class order, and the spray started misfiring before anyone noticed. The model package also ships a `class_indices.json`; Lambda asserts the indices match `CAT_NAMES` on cold start, so a retrained model with a shifted class order fails loudly rather than silently misfiring:

```typescript
// On cold start, assert that class_indices.json matches CAT_NAMES order
const classIndices: Record<string, number> = JSON.parse(classIndicesJson);
CAT_NAMES.forEach((name, i) => {
    if (classIndices[name] !== i) throw new Error(
        `Class index mismatch: expected ${name}=${i}, got ${classIndices[name]}. Full class_indices.json: ${classIndicesJson}`
    );
});
```

`classIndicesJson` is read from the model package in S3 during the cold-start initialisation path, alongside the model weights.

## The second opinion

SageMaker is fast, but it's a black box. It tells you probabilities; it won't tell you *why*. That's fine for day-to-day operation, but it makes diagnosing drift or miscalibration genuinely difficult. If the model starts getting Kappa wrong — he's very dark, long-coated, easily confused with Boots in poor light — you won't know until you go looking.

Claude 3 Haiku with vision gives you something different: a structured audit trail. The prompt constrains the response to a fixed schema and a fixed set of names, including an explicit no-cat option:

```typescript
const CLAUDE_SYSTEM_PROMPT = `You are a cat identification system for a home monitoring project. Identify which cat is visible in the image from the following list:

- Boots: solid black, short-to-medium coat, bright green eyes. This is the neighbour's cat and the primary target of the system.
- Chi: Maine Coon, dark brown/black tabby pattern, long fluffy fur, green eyes.
- Tau: black and white tuxedo pattern (black head/back, white chest/belly), short hair.
- Kappa: very dark brown/near-black, long fluffy fur (son of Chi and Tau, inherited Chi's coat).
- Mu: Maine Coon, ginger/cream coloured, long fluffy fur, regal expression.
- Wolf: a grey/white stuffed toy cat — not a real cat.
- NoCat: no cat clearly visible in the image.

No code fences, no preamble. Respond with a single JSON object only, no other text. Use only the exact names listed above.
{"cat": "<name>", "confidence": "high|medium|low", "reasoning": "<one sentence>"}`;
```

The names in the prompt must stay in sync with the `CAT_NAMES` list the validator checks against — drift between the two silently turns valid answers into validation drops. The response schema is intentionally minimal. Cat name, confidence band, one sentence of reasoning. If Claude returns invalid JSON, an unrecognised name, any extra top-level fields, or reasoning over 300 characters, the result is dropped and a log event is emitted. Validation is a few lines:

```typescript
function validateClaudeResult(result: unknown): ClaudeResult {
    if (result === null || typeof result !== 'object') throw new Error('Result is not an object');
    const r = result as Partial<ClaudeResult>;
    if (!CAT_NAMES.includes(String(r.cat))) throw new Error(`Unrecognised cat: ${r.cat}`);
    if (!['high', 'medium', 'low'].includes(String(r.confidence))) throw new Error(`Invalid confidence: ${r.confidence}`);
    if (typeof r.reasoning !== 'string' || r.reasoning.trim() === '' || r.reasoning.length > 300) throw new Error(`Invalid reasoning`);
    const knownKeys = new Set(['cat', 'confidence', 'reasoning']);
    const extraKeys = Object.keys(r).filter(k => !knownKeys.has(k));
    if (extraKeys.length > 0) throw new Error(`Unexpected fields: ${extraKeys.join(', ')}`);
    return r as ClaudeResult;
}
```

JSON parse and validation failures throw inside `invokeClaudeVision` and are caught by the same `.catch` handler, so a malformed response is treated the same as an API failure. The valid result ends up in DynamoDB alongside the SageMaker probabilities, timestamped, attached to the S3 image key.

```json
{"cat": "Boots", "confidence": "high", "reasoning": "Solid black coat, bright green eyes visible in upper frame"}
```

## The interesting part

SageMaker takes tens to low hundreds of milliseconds (including the Lambda-to-endpoint round-trip); Claude 3 Haiku with vision typically takes a few hundred milliseconds. The real concern isn't absolute latency — it's that Claude must never delay the deterrent.

In a synchronous API Gateway integration, the HTTP response isn't sent until the Lambda handler finishes. An ESP32 POST would hang open until Claude resolved — which is why ordinary catcam triggers never set `?claude=1`. For normal live triggers `claudePromise` resolves immediately as `undefined` and the handler finishes as fast as the SageMaker call plus the deterrent call plus the DynamoDB write.

The `?claude=1` flag is only honoured for authenticated manual review sessions — think: examining a specific tricky frame after a false positive, or auditing a batch of stored images before a retraining run — where a slightly longer response time is acceptable and you want the reasoning in the DynamoDB row. Live ESP32 triggers never set this flag, so Claude never runs on the live path and cannot delay the spray. The API Gateway route requires a valid authorizer token, so untrusted callers cannot force LLM spend by probing with `?claude=1`. The flag check happens after the authorizer has already validated the caller:

```typescript
const isAuthed = event.requestContext.authorizer?.principalId;
const claudeInfer = Boolean(isAuthed && event.queryStringParameters?.claude === '1');
```

One guard worth stating: review sessions examine stored images, and `maybeTriggerDeterrent` is guarded upstream so a review request will not fire the spray.

The setup for both calls happens before the awaits. `invokeClaudeVision` is wrapped with a timeout so a slow Haiku response doesn't extend Lambda duration beyond what a review session is expected to tolerate — the function timeout is 2 seconds, and Claude is capped at 1 second, which leaves comfortable headroom for SageMaker, the deterrent call, and the DynamoDB write in the worst case. A bare `Promise.race` isn't enough here: the race doesn't cancel the losing promise, so a timed-out Claude request would carry on in flight — costing money — with its result discarded. Wiring an `AbortController` through so the timeout genuinely cancels the request fixes both problems at once:

```typescript
// Abort the in-flight HTTP request when the timeout fires
async function invokeClaudeVision(imageBuffer: Buffer, signal: AbortSignal): Promise<ClaudeResult | undefined> {
    const client = await getAnthropicClient();
    const response = await client.messages.create({ /* ... */ }, { signal });
    // parse and validate response ...
}

function withTimeout<T>(
    factory: (signal: AbortSignal) => Promise<T>,
    ms: number,
    label: string
): Promise<T> {
    const controller = new AbortController();
    let timer: NodeJS.Timeout | undefined;
    const timeout = new Promise<never>((_, reject) => {
        timer = setTimeout(() => {
            controller.abort();
            reject(new Error(`${label} timed out after ${ms}ms`));
        }, ms);
    });
    return Promise.race([factory(controller.signal), timeout])
        .finally(() => { if (timer) clearTimeout(timer); });
}
```

The timer is cleared in the `finally`, so nothing leaks across invocations, and the abort means a timeout stops the Claude spend rather than merely ignoring it. With that in place, both calls start before either await:

```typescript
// Start both calls simultaneously — SageMaker's result is acted on first,
// Claude's arrives later and is only used for logging
const sagemakerPromise = sagemakerClient.send(invokeCommand);
const claudePromise = claudeInfer
    ? withTimeout((signal) => invokeClaudeVision(imageBuffer, signal), 1000, 'Claude').catch((err) => {
        logger.error('Claude inference failed or timed out, continuing without it', { error: err });
        return undefined;
    })
    : Promise.resolve(undefined);
```

Then the deterrent decision happens as soon as SageMaker resolves, before we wait for Claude at all:

```typescript
// Await SageMaker and trigger deterrent immediately
const sagemakerResponse = await sagemakerPromise;
await maybeTriggerDeterrent(sagemakerResponse);

// Now await Claude — its result goes in DynamoDB alongside SageMaker's,
// but it never gates the spray
const claudeResult = await claudePromise;
try {
    await writeEvent({ sagemakerResponse, claudeResult });
} catch (err) {
    logger.error('writeEvent failed, continuing', { err });
}
```

`maybeTriggerDeterrent` is awaited, not fire-and-forget — it makes an out-of-band call to the deterrent hardware, and an un-awaited network call can get cut off if Lambda freezes the execution environment the moment the handler returns. (Lambda freezes rather than terminates, which is why the call can be silently abandoned rather than cleanly errored.) Awaiting it here doesn't delay Claude, which is already in flight on `claudePromise`; it just makes sure the spray command has actually left the container before we move on.

`writeEvent` is wrapped in a try/catch so a DynamoDB failure doesn't propagate an error response to the ESP32. If `writeEvent` throws after the spray has already fired, the correct behaviour is to return success and log the write failure — not to return an error that might prompt the device to retry and spray again. Awaiting the write before returning is an intentional choice: it trades a few extra milliseconds of tail latency for at-least-once persistence. An alternative would be to return before the write and forward the event via an async channel (e.g. EventBridge) if lower tail latency ever becomes a priority.

The total Lambda duration when `?claude=1` is active is roughly `max(SageMaker + deterrent call, Claude) + writeEvent` — e.g. 150ms + 200ms on the SageMaker side against a 400ms Claude call comes out at about 400ms plus the write. You pay for concurrency, not the sum of both models in series.

(The SageMaker call itself has no per-request timeout beyond the 2-second Lambda backstop; per-request timeouts and keep-alive agents on the AWS SDK HTTP client would be the tidier production choice.)

SageMaker always gets the image buffer; when `claudeInfer` is true, Claude receives the same buffer at the same time. SageMaker gets it wrapped in TF Serving's `instances` format (`{"instances": [{"b64": "..."}]}`); Claude gets it as a base64 image block in a Messages API call. SageMaker's failure is allowed to propagate, because without that result there's nothing to act on; Claude's is swallowed (or timed out), because its absence changes nothing for the spray.

![A detective's cork board covered in near-identical photographs of a sleek black cat, one photo ringed with red thread, illuminated by a desk lamp in a shadowy room](images/parallel-inference-sagemaker-claude-vision-2.png)

## What discrepancy logging actually tells you

The catcam-events table records both results when Claude is active — only rows where `claudeInfer` was true will ever have a Claude result. Each row also stores a `claude_status` field (`ok`, `timeout`, `error`, or `skipped`) and an error code when relevant, so downstream queries don't have to infer why a Claude result is absent. That produces rows like:

| SageMaker Boots Prob | Claude Cat | Claude Confidence | Claude Status |
|---|---|---|---|
| 0.94 | Boots | high | ok |
| 0.87 | Boots | medium | ok |
| 0.71 | Kappa | high | ok |
| 0.62 | NoCat | low | ok |

The third row is interesting. SageMaker is 71% confident it's Boots. Claude is highly confident it's Kappa. That's not noise — that's disagreement with a reasoning trail attached.

At the moment I query catcam-events weekly to look for systematic disagreement and tag spans for retraining. The next step is to automate this: a Lambda triggered by a DynamoDB Stream (or on a schedule) detects rows where `probs[0] ≥ 0.7` and `claudeResult.cat != "Boots"` with medium or higher confidence, publishes a custom CloudWatch metric, and writes matches to a separate `catcam-discrepancies` table. A CloudWatch Alarm on that metric then fires when the pattern becomes systematic. For now, the weekly query is enough.

What you're looking for is systematic disagreement, not individual frames. One image where a very dark Kappa gets tagged as Boots by SageMaker is a lighting artefact. Fifty images over two weeks showing the same pattern is a calibration problem. The remediation is straightforward: collect more Kappa-in-low-light images, retrain, and verify the class-index assertion still holds.

## The Anthropic client stays warm

Lambda cold starts are expensive when you're fetching secrets. The Anthropic API key lives in SSM Parameter Store rather than an environment variable — it keeps the key out of the Lambda console and avoids the per-secret cost of Secrets Manager for something fetched rarely. The first invocation has to make an SSM call (with KMS decryption for a SecureString) before it can make a Claude call. The SSM client itself is also module-level, so it too survives across invocations — as do the SageMaker and DynamoDB clients, which are worth caching at module scope for the same reason.

The solution is to cache the Anthropic client across invocations:

```typescript
// Cached Anthropic client — resolved once per Lambda container lifetime
let anthropicClient: Anthropic | undefined;

async function getAnthropicClient(): Promise<Anthropic> {
    if (anthropicClient) return anthropicClient;

    const result = await ssmClient.send(new GetParameterCommand({ Name: paramPath, WithDecryption: true }));
    const apiKey = result.Parameter?.Value;
    if (!apiKey) throw new Error(`Anthropic API key not found at SSM path: ${paramPath}`);
    anthropicClient = new Anthropic({ apiKey });
    return anthropicClient;
}
```

Lambda container reuse means the second and subsequent invocations in a warm container skip the SSM call entirely. It's the same pattern as caching a database connection — obvious in retrospect, easy to miss when you're thinking about the inference logic rather than the initialisation path. The one caveat is key rotation: if the API key is rotated, the cached client will start returning 401s. Catch auth errors in `invokeClaudeVision`, clear `anthropicClient`, and let the next call re-fetch from SSM.

Since only authenticated callers can trigger Claude at all, the blast radius for unexpected spend is limited. Still, it's worth setting a budget alert on the Anthropic account so a misconfigured review script hammering `?claude=1` doesn't go unnoticed.

## What this is, and what it isn't

The SageMaker model makes the deterrent decision on live ESP32 triggers; Claude provides the audit trail for manual review sessions, running concurrently with the rest of the handler but never gating the spray.

The broader pattern — fast primary model for the action path, slower general model for the audit trail, both started simultaneously, only the primary gating the decision — applies anywhere you want explainability without sacrificing latency. It works less well on high-QPS paths where doubling inference cost matters on every request, or in cases where the auditor's disagreement should actually *block* the action rather than just record it. The main thing to get right is being honest about what "concurrent" means in a synchronous Lambda integration, and designing the query-parameter gate so the slower path is only taken when you've explicitly asked for it.

Two models walk into a Lambda. The first one fires. The second one explains why.

---

*Martin Harris writes about connecting real hardware to abstract cloud infrastructure. If you've ever soldered something to an API Gateway, you're in the right place.*

{{donate}}