Senior Software Engineer · Apr 2025 – Oct 2025
A ten-agent pipeline that can be replayed
Fan out the reads, serialize the writes
Built for an employer, described with their approval. The product, its clients and its internals stay unnamed — the architecture carries the signal, those names don’t.
Ten agents read the same batch in parallel and exactly one thing writes. Letting each agent own its slice of state feels more autonomous and is how most agent systems get built — it also means every new agent multiplies the interleavings you have to reason about, and the failures arrive as corrupted relationships at 3am rather than as a stack trace.
Every write goes through a content hash first. Re-running the pipeline over the same source events produces the same graph, which means Postgres is a cache rather than the record — it can be wiped and rebuilt from source. That one property at the storage layer is what turns a pipeline you are afraid to re-run into one you can.
A Redis-only raft collects incoming messages until a threshold or timeout before the ten-agent fan-out ever runs. Batching amortises the Gemini call across every message in the batch instead of paying full model latency and cost per message — the fan-out pattern only works economically because this stage exists in front of it.
384-dimension embeddings — an order of magnitude smaller than the 3,072-dim used in the conversation-intelligence platform, deliberately, because clustering short chat messages into topics doesn't need the resolution that distinguishing fine semantic shades across long transcripts does. The single global 70% cutoff is the rough edge: it serves the middle of the topic-density distribution well and the tails poorly.
Five query operations sit on top of the same 27-table schema — semantic search, three demand/engagement signals, and a profile lookup. None of them is a hand-written endpoint: declaring a query operation on the storage raft is what makes the gateway expose it as an MCP tool automatically, so an external LLM can ask what a chat group cares about without anyone writing that route.
The problem
Chat events arriving continuously, needing to become a graph: who is talking, about what, how they relate, what they care about. Ten different analyses per batch, all wanting to write.
The naive shape is to let each agent own its slice of state. It feels more autonomous, and it is how most agent systems get built. It also means every new agent multiplies the number of interleavings you have to reason about — and the failures show up as corrupted relationships at 3am, not as a stack trace.
Fan out the reads, serialize the writes
Before the ten-agent fan-out runs at all, a Redis-only batching layer collects incoming messages until a batch threshold or timeout is hit. That step is why the fan-out is affordable — one model call per agent per batch, not per message. The story usually gets told starting at "ten agents read the same batch in parallel," which skips the part that makes it economical.
Seven model-backed agents — sentiment, embedding, toxicity, spam, emoji, topic, relationship — each make exactly one model call per batch and return a result. They are stateless. They touch nothing. Because they share no state, they parallelize perfectly, and adding an eighth costs nothing conceptually.
Three coordination agents split the rest of the work, each with a specific job: one merges every model agent's result into the in-memory graph, one maintains the vector index, one assembles context for the downstream query layer. The first of those — the graph merger — is the only writer. Every result merges through it.
No locks. No partial merges. No ordering bugs. The pattern is old and it is the entire reason this system was debuggable.
The in-memory graph is per-batch, not global — it captures one batch, gets persisted, and is discarded. That bounds memory and means a bad batch can't poison the next one.
Content-hash idempotency
Every write is keyed by a hash of its source content. Re-running the pipeline over the same input produces byte-identical output.
The consequence is bigger than it sounds: Postgres became disposable. We could wipe the database entirely and rebuild it from source events, deterministically. Debugging stopped being archaeology — instead of reasoning about which records got half-processed when stage six failed, you fix the bug and replay.
The discipline it demands is being ruthless about what enters the hash. Anything non-deterministic — timestamps, model temperature, retry counts — has to stay out, or every run looks like a change and the property silently stops holding. That is the failure mode to watch: idempotency doesn't break loudly, it just quietly stops being true.
In nine months of production, "just replay it" resolved more incidents than any dashboard did.
Retrieval and clustering
384-dimension embeddings — an order of magnitude smaller than the 3,072 I used elsewhere, and deliberately so. That system needed to distinguish fine semantic shades across long-form transcripts. This one needed to cluster short chat messages into topics. Bigger vectors would have cost storage and query time to resolve distinctions that don't exist at this granularity.
Topic assignment uses a 70% similarity threshold: above it, a message joins an existing cluster; below, it starts a new one. Backed by a 27-table schema with 49 indexes and 5 views — three numbers describing the shape of one schema, sized for a graph that's queried far more than it's written to.
Pipelines as infrastructure
The part I find most interesting architecturally. Pipelines are declared as infrastructure-as-code over custom resource types — a stream, an indexing-and-query primitive, an agent — rather than as application code.
A new pipeline ships as a stack file. It deploys without recompiling any backend service. The platform's query gateway picks up the declared query operations automatically and exposes each as an MCP tool, so an external LLM can ask "what are people talking about in this group" and get a structured answer without anyone writing an endpoint.
Concretely, that's five tools: semantic_tree_query, what_people_are_talking_about,
what_people_care_about, what_people_want, and user_details. An external LLM calls
what_people_care_about directly and gets a structured answer back — no bespoke endpoint, no
schema explanation, because the query operation was already declared as part of the pipeline's
infrastructure rather than bolted on afterward.
That is the payoff of treating a pipeline as a resource rather than a program: extending the system stops requiring a deploy of the system.
The graph isn't only queryable, it's visible: a small Next.js UI reads directly from Postgres and renders the social graph, so the end of this pipeline is something a person can look at, not just something an agent can ask about.
What I'd do differently
The clustering threshold is a single global constant. Different communities have genuinely different topic densities, and 70% is a compromise that serves the middle of the distribution well and the tails poorly. Per-stream calibration would have been a day's work and I didn't do it.