Production agents: the hard parts

Most people are familiar with how easy it is to build an agent or skill in Claude or another leading provider, both technical and non-technical builders. You can add in tools, connect it to your Google Workspace, Slack, or whatever connections you have, and it will work "perfectly", at least for a demo.

Agent Delivery at Rightbrain, in practice, means taking agents from prototype to production: scoping the right use cases, architecting them to be efficient and observable, and getting them robust enough to run reliably at scale once real people and real edge cases hit them. So the gap between a great demo and a production-grade agent is the thing I deal with every day, and it is where almost all the hard parts live.

After a few runs, or when you hit your first issues, things start to go off-piste. Before you know it there is a sense of losing control, a lack of transparency, and it is easy to become disillusioned, and that is before you open it up to other people in your business and encounter all the edge cases that happen in production. There are also the escalating costs you don't want to tackle without losing the quality of output you spent hours or days achieving in early testing. We see this time and again with our customers, and it boils down to not building the agents in a way that is suitable for a production environment, at scale.

Brute forcing increases costs

Tokenmaxxing is definitely not the flex it was a few months ago. It means letting the model do whatever it takes to get to an answer, burning through tokens and API costs, spinning up unnecessary loops and reasoning steps, and turning something that should be deterministic into something erratic that needs minutes of thinking time. It is really a symptom of a lack of precision in either the prompt or the configuration of the agent. Use any of the frontier models from the big three (Anthropic, Google, and OpenAI), and they will usually paper over the gaps and produce something that looks sensible. But that relies on luck, and at production volumes the misses start to add up.

Using the right architecture

The basic rule of thumb: the simpler and more deterministic the agent and tasks, the better, so reduce the agent's thinking and reasoning as much as possible. If you are running Opus 4.8 with an orchestrator that needs several reasoning steps, or re-ingests the same data on every turn, that should be split into sequential tasks that are observable and controllable independently of each other, but that run together according to well-defined logic rather than the model choosing the path each time. Only then do you get full visibility and the ability to optimise cost at every step. Determinism wins every time.

The expenses processing agent we built in-house for finance teams is a good example. It uses a more powerful model for orchestrating, but a small model classifies, extracts, and validates perfectly well at a fraction of the cost, removing around 90% of the unnecessary reasoning. It could be done by the orchestrating agent, but then you lose control and transparency.

The power of being model agnostic

Fallback model

This is an essential part of agents and tasks in production. If the primary model fails (which is more often the case than you might expect), you need a similarly capable model from a different provider to seamlessly carry out the task. Even the major providers go down or hit capacity limits without warning, and managed services can enforce global usage quotas that fail your task at seemingly random moments. None of it is your fault, and none of it is something you can prevent from inside a single provider, which is exactly why a fallback on a different provider matters.

Avoiding vendor lock-in

Not being tied to a model provider is not only beneficial, it is essential for any enterprise today. It matters in production, but also for evaluation, because being able to run the same agent across different models makes it far easier to replicate or simulate how it will actually behave once deployed.

We are also entering a period where safety and regulatory pressure can change a model's availability at short notice. A provider can restrict access to a model, or pull a version entirely, on grounds that have nothing to do with your use case, and if your agent is built around that one model, you inherit that risk with no say in it. Open-source models need to remain an option for many businesses, as data becomes more valuable and running private instances enters the radar.

Transparency wins

Ask most people who have built a Skill in Claude to tell you what the steps are in the reasoning or processing, and most will return your question with a blank look. Even the ones who can describe it in theory usually cannot tell you whether the agent actually executed those steps in practice. That is the dangerous part. When a step is missing or a tool quietly fails, the model does not stop and flag it; it carries on and returns the most plausible-looking result. You get a confident answer that looks right but has skipped something important, and without visibility into the run you have no way of knowing.

This is why, on every agent we put into production, you have to be able to explain and review the execution flow for any given run, not the intended flow, the actual one. Once you can see the cost, latency, and inner workings of each step, the fixes are usually high value and low effort: not just model choice, but prompt engineering, tool use, or removing a tool and replacing it with a configured, deterministic task. Often a few hours of work takes a significant chunk out of the running cost.

Your team in control, supported however you need

For many enterprises, once you understand what needs to be done in production with your agents, the natural next step is to manage them in-house: learn from edge cases, handle them quickly, and run your agents like you do your own software stack. That is also where the care you took over the initial build pays off, as your own team becomes the one continuously improving the agent and driving it down the cost curve.

In the expenses agent, the orchestrator does real work when it outputs the extracted receipt data and appends it to a Google Sheet, or pushes it through a tool into a third-party system like Xero. Today, that justifies a more capable model with enough reasoning power to overcome edge cases. Over the coming months, as we discover those edge cases and harden the prompt and instructions across iterations, we can move that step to a smaller or open-source model, which on this build points to roughly another ten-fold reduction in cost. You earn the cheaper architecture as you become more precise and efficient.

If you are trying to get agents working properly inside your own workflows, that is exactly what we do, so do get in touch.