Beyond the wrapper: building resilient multi-agent AI workflows in production

Most "AI agent" projects work on the first try and then never work reliably. The demo is a model with tools in a loop, and that is genuinely all a demo needs. The gap between it and something you would let touch a production database is not model quality — it is everything around the model.

This is the shape we build agentic systems to, and why each part is there.

Start with the boundary, not the prompt

The first artefact is not a prompt. It is a written statement of what the system may do without asking, what it may do with approval, and what it may never do.

Three bands, decided in discovery and written down:

The third band matters most and is the one people skip. A capability that is merely discouraged in the prompt is a capability the system has.

// The executor decides, not the model.
const decision = policy.evaluate(tool.name, args, context);
 
if (decision === 'deny') throw new ToolNotPermittedError(tool.name);
if (decision === 'approve') return await queueForApproval(tool, args, context);
return await tool.run(args, context);

Tools are an API you are designing for an unreliable caller

Treat every tool as a public API whose consumer will pass the wrong arguments, call it twice, and call it in the wrong order — because it will.

1. Idempotent by construction

   Every mutating tool takes an idempotency key derived from the intent, not from the call. A retried create_refund with the same key returns the original refund rather than making a second one. This is the single highest-value property in the whole system.

2. Narrow, typed arguments with validation at the edge

   Not run_query(sql). find_orders({ customerId, status, limit }). A tool that accepts arbitrary SQL has handed the model your database, and no prompt undoes that.

3. Errors that describe the fix

   "Invalid date" leaves a model guessing. "from must be ISO-8601, got 03/04/2026" gets the next attempt right. Error text is prompt engineering with a much better feedback loop.

4. Return the minimum

   A tool returning a 40-field object spends context on 38 fields nobody used, and buries the two that matter. Project before returning.

The loop needs bounds it cannot argue with

An agent loop without hard limits does not fail — it spends.

const limits = {
  maxSteps: 12,
  maxToolCalls: 20,
  maxTokens: 120_000,
  maxWallClockMs: 90_000,
  maxCostUsd: 0.75,
};

Every one of those is enforced by the runner rather than requested in the system prompt. When a limit is hit the run stops and reports why — it does not silently truncate, and it does not hand back a partial answer that looks complete.

Multi-agent is a decomposition decision, not a feature

Splitting one agent into several is worth doing when it reduces the search space. It is not worth doing because the architecture diagram looks better.

The critic pattern is the one most worth its cost, and only under a condition: the critic must be able to check something the generator cannot fake. Compiling the code, running the test, validating against a schema, re-querying the source. A second model asked "is this good?" is two opinions, not a verification.

Assume the model is wrong, and design for it

The interesting design work is in the failure paths.

1. Every run is resumable

   Persist the step log — messages, tool calls, results — so a crashed or rate-limited run continues rather than restarting. Restarting is not just slow, it re-executes side effects, which is why idempotency comes first.

2. Structured output is validated, then repaired once, then failed

   Parse against the schema. On failure, hand the validation error back for exactly one retry. A second failure is a real failure — an unbounded repair loop is the expensive pattern above wearing a different hat.

3. Retrieved content is data, never instruction

   Text a model retrieved can contain "ignore previous instructions". Keep retrieved content in a clearly delimited role, never concatenated into the system prompt, and remember the policy layer is what actually stops the damage.

4. A degraded answer beats a confident wrong one

   When a tool is unavailable, the honest reply names what is missing. A model asked to proceed without its data will produce something plausible.

Cost and latency are architecture, not tuning

Three levers, in descending order of effect:

1. Do not call the model

   Cache by semantic similarity for repeated questions, and route deterministic work — a lookup, a calculation, a format conversion — to code. The cheapest token is the one not generated.

2. Call a smaller model

   Routing, classification, extraction and reformatting rarely need the frontier model. A router picking between a small and a large model on a measured confidence signal is usually the largest single cost reduction available.

3. Send less context

   Context is quadratic in attention and linear in price. Summarise the step log rather than replaying it, project tool results, and drop retrieved chunks that scored poorly instead of padding to a fixed count.

What to measure

An agent system without evaluation is a system whose behaviour changes when a vendor ships a model update and nobody notices.

Keep the evaluation set versioned alongside the prompts, and run it in CI. A prompt change is a behaviour change, and it deserves the same gate as a code change.

The short version

1. Write the policy before the prompt

   Three bands, enforced by the executor. The never-band is a tool that does not exist.

2. Make every mutating tool idempotent

   It is what makes retries, resumption and crash recovery safe rather than dangerous.

3. Bound the loop in the runner

   Steps, calls, tokens, wall clock, cost — and detect repetition explicitly.

4. Only add an agent when it narrows the problem

   And only add a critic when it can check something the generator cannot fake.

5. Design the failure paths first

   Resumable runs, one repair attempt, retrieved content as data, degraded answers over confident wrong ones.

6. Evaluate in CI, and watch p95 steps

   A prompt change is a behaviour change.