Notes on production systems

Writing

A numbered series of teardowns, engineer to engineer. Lead with the problem, one concrete number, name the cost as well as the win. All 16 are written and readable here; the dates are when each goes out on LinkedIn.

• 01Aug 18scheduledContent-hash idempotency — making an agent pipeline replayableMost LLM pipelines can't be re-run. One decision at the storage layer made ours fully replayable and turned Postgres into a disposable cache.
• 02Aug 20scheduledCutting LLM pipeline compute ~94% — ECS on EC2 vs FargateIt wasn't a clever model choice. It was noticing that one capacity strategy across three workload shapes means overpaying on at least two.
• 03Aug 25scheduledFan-out reads, single-writer stateTen agents processing the same batch in parallel. The pattern that keeps them from corrupting shared state is older than any of this.
• 04Aug 27scheduled7 queues, 7 DLQs: what failure handling actually looks likeOne queue with a type field is simpler right up until one slow stage starves every other one.
• 05Sep 1scheduledExposing pipeline queries as MCP tools automaticallyThe integration you don't write is the one that can't drift.
• 06Sep 3scheduledA promotion ladder that never demotesMonotonicity turned a graph traversal into a threshold comparison, and made every downstream cache safe.
• 07Sep 8scheduledPipelines as infrastructure-as-codeA new pipeline shipped as a stack file and deployed without recompiling any service.
• 08Sep 10scheduledWhere the latency actually goes in a sub-500ms voice pipelineThree different numbers get rounded to "half a second" — a hard budget, a tuned target, and a median. Only one of them is a promise.
• 09Sep 15scheduled3,072 dimensions vs 384: choosing embedding size by use caseI used both, eight months apart, and the difference wasn't budget.
• 10Sep 17scheduledMulti-tenant isolation with Postgres RLS — why single-deployment wonPer-tenant deployments are easier to reason about and multiply every operation you'll ever perform.
• 11Sep 22scheduledA 7-stage speaker-identification cascadeOrder the checks by confidence, not by cleverness, and the expensive one runs almost never.
• 12Sep 24scheduledAutoscaling on backlog-per-task instead of CPUQueue depth alone is the wrong signal and CPU is a lagging one. The honest unit is messages per worker.
• 13Sep 29scheduled24 tools in one function-calling registry: organizing a namespacePast about a dozen tools, the model's problem stops being capability and starts being selection.
• 14Oct 1scheduledAgent memory across sessions: reprocessing vs retention windowsA window forgets by recency. Reprocessing refuses to forget at all. Both are sorting on the wrong axis.
• 15Oct 6scheduledWhat "production-ready" means for an agent systemFour questions. None of them are about the model.
• 16Oct 8scheduledNine months of agent infrastructure: what I'd do differentlyThree things I got right for reasons I couldn't measure, and one I'd rebuild.