What I built in seven weeks, who it is for, and the one claim it makes.
This is the post to read before the other two on this blog. taidle is a knowledge base with an LLM agent over it, built so that an agency can run it for many clients. The two earlier posts are debugging stories from inside it. This one is the outside: what it is, who operates it, what runs today, what is only designed, and the one decision I made by hand that shaped everything else.
Answers you cannot check
The original idea was agent-to-agent. I run local agents, Claude Code and a command-line assistant among them, and I wanted one place they could all read from and write to: a single source of truth for what I know, instead of a memory scattered across each tool's own context. taidle was to be that place, and the agents were to be its first clients.
That plan meets a fact about agents. An agent will do something unexpected. It is a statistical model predicting the next token, and I cannot claim to make that impossible; I know of nothing that can. What I can say is where the cost lands. One surprising reply is cheap. The expensive thing is the second time, and the tenth, when the same mistake comes back and you still cannot tell what the agent read, what it skipped, or which rule it was following, so you cannot make the next occurrence rarer or smaller. The loss is not the error. The loss is the error recurring where no one can check it.
I had already lived that as a user. I kept asking assistants about my own material and getting answers I could not verify. Not wrong, necessarily. Unverifiable. When one was wrong I could not tell whether the model had read the note I meant, a different one, or nothing at all, so I could not fix anything. I could only ask again with different words.
So I wrote down one claim and refused to let anything violate it:
Every reply renders its route. The context the model actually saw, the tools that fired, the policies that were checked.
Not logged for debugging. Rendered, on screen, while the reply is still streaming.
The claim is deliberately narrow. A trace can prove what the system did and what was put in front of the model. It cannot prove why the model produced a token, and a product that implies otherwise is selling a story about interpretability that the mechanism does not support. It also does not claim to prevent the mistake. It claims that the mistake, when it happens, leaves a route someone can read. Bounding the claim is what makes the rest of it worth anything.
What taidle is, from an agency's seat
taidle is not a consumer app. The way it is meant to be used is by an agency that builds AI assistants for its own clients:
- One account per client. Each account holds that client's knowledge base: their notes, the facts the system has extracted from conversations, the conversation memory itself, and a set of behaviour skills that decide how the assistant acts. In this build a knowledge base belongs to one account. A shared workspace across accounts is a seam the design reserves and does not yet build.
- The agency's own front end. taidle ships its own interface for reading a reply's route. The surface a client's customers or staff see, a chat box on the client's website or an onboarding bot inside Slack, is the agency's to build. It connects to taidle's streaming endpoint, which sends each step of a reply as it happens, so the agency's surface can show the route too.
- Reachable from agents people already run. taidle serves its own tools over MCP, so a client's assistant can be called from Claude Code or a local CLI. It does not consume third-party MCP servers; that would change what "trusted content" means inside a run, and I recorded why it waits.
- Roles with ceilings. A guest, a member and the owner of an account get different limits on how far the agent may loop on one reply. The numbers are set per role in configuration, not hard-coded.
One reply, six stages
Every reply runs through the same six stages, and every stage writes a committed row before the next one starts.
- Gate. A cheaper model decides whether this turn needs a search at all. Its verdict and its reason are a recorded step, not a hidden branch.
- Retrieve. Four separate stores are searched: behaviour skills, extracted facts, past conversation records, and the client's notes, the last one by vector and keyword search fused together.
- Assemble. What was found is packed into token-bounded bands, so a large tool result cannot push the conversation out, and the exact request is hashed.
- Loop. The model is called, tools are dispatched, and the loop stops on a token or iteration ceiling instead of running away.
- Check. Every tool call is checked against the role's policy. Content from the web marks the run as tainted for the rest of its life, and a tainted run cannot write to memory without a human approving it.
- Distill. Afterwards, in a separate run, durable facts are extracted from the conversation and written back.
The route a client sees is this list, filled in with what actually happened.
What runs today, and what is only designed
taidle is not deployed. There is no production environment, and I would rather say that in the second paragraph of the section than let a reader assume otherwise. Everything in the left column runs on one machine and is exercised by the test suite on every push. Everything in the right column is written down in the repository as a plan.
| Runs today, on one machine | Designed and recorded, not built |
|---|---|
| One Go binary with the web interface embedded | The binary on a managed host, deployed from a container image |
| PostgreSQL with pgvector, versioned migrations | Managed PostgreSQL |
| An S3-compatible object store for backups and evaluation artifacts | A cloud object store, by environment change only |
| A boot gate: in production mode the binary refuses to start unless the exact configuration it carries was evaluated and promoted | Already real, and environment-independent |
| A backup that streams to the store and a restore drill into a scratch database | The same drill on a schedule |
The one design choice bridging the columns: every item on the right is an environment change, never a code branch. The day it deploys, it inherits every proof the left column has already earned.
The numbers
| Build time | 7 weeks |
| Commits | 1,312 |
| Lines of Go | 239,000, 61% of them tests |
| Behavioural assertions replayed on every push | 73 |
| Provider spend per CI run | $0, and CI fails if any call reached a provider |
| Architecture decision records | 16, amendments included |
Every number traces to a committed artifact: a ledger line, a generated report, a decision record's own measurement. Nothing here is estimated for effect.
What I chose, and what it cost
The obvious way to build this is to pick an agent framework. That is what most teams do, and for most products it is right. A framework optimizes for speed to a working demo and for a vocabulary other engineers already know.
taidle needed something a framework has to hide. When the trace is a debugging aid, it is fine that the loop's state lives inside the library's objects, that the bytes sent to the provider are assembled by code you did not write, and that retries happen somewhere you cannot observe. When the trace is the product, each of those is a hole in the claim. I cannot render a route I cannot see, and I cannot fingerprint bytes I did not assemble.
So the execution engine is written from scratch in Go. Every tool call, retrieval decision and policy check is a typed row in the database, streamed to the interface as it happens.
Here is what that cost, stated plainly:
- The engine before the product. Weeks on execution, provenance and replay while the application itself was a shell. On a seven-week project that is most of the budget.
- Real money. Re-recording model responses every time a prompt or a rendering constant moved, until I learned to separate what had actually changed from what had not.
- A stall. A period where I stopped changing things because small edits kept breaking unrelated parts and the failure messages lied about why. It is the part of the project I am least proud of and learned the most from.
I would make the same decision again. I would make it with the lessons from that stall in place on day one.
The rule
A small claim you can prove beats a large one you cannot.
"Every reply shows what the model was given" is small, and every line of taidle exists to keep it true. "The assistant explains its reasoning" is large, and no mechanism I know of can back it. The first claim is what a client can check on screen. The second is what gets a product into trouble the first time someone checks.
Where to read next
The two earlier posts are what happens when you can actually measure a system like this. Check Your Metric's Ceiling is about a benchmark that could not report anything higher than the number I was about to spend weeks improving. My Vector Search Had No Vectors is about a retrieval channel returning noise while every test stayed green.
The next posts go the other way, toward the basics: what retrieval-augmented generation is, how an engineer builds one, and where the trade-offs sit.
One thread from a longer set of engineering notes on building taidle — a knowledge base with an LLM agent over it, where every reply renders the route the system actually took. The essays and the 16 architecture decision records are at github.com/matthewhoung/taidle-labs.