Agentic Chaos

I recently wrote about how Agents are taking us back to our engineering roots. Whilst the fundamentals have never gone away (maybe we just got better at hiding them in our default tooling), Agents now force us to re-consider just how we're addressing some of these basics.

I think Agents are here to stay, and if they're going to be a permanent fixture in production systems, making real-world decisions that have very real-world consequences, then we need to come up with real-world solutions to providing the degree of engineering rigour we've become used to.

And the Agentic world is a messy one: tools fail, models degrade, integrations return strange data, retries compound and the delegation of autonomy needs serious governance. Agents bring Chaos!

So if Agents are bringing the hard engineering problems back again, then resilience engineering/testing has to become part of the platform (and the standard Agent development process) - not a one-off exercise condemned never to be repeated again.

The bluff of the successful demo

I previously wrote about how the era of the AI demo is closing. Let me dig into one component of that further.

The demo went great, it called the right tool, produced the right answer and streamed something that looked shiny: you breathe a sigh of relief!

But production? It will ruthlessly expose any and all shortcomings in that shiny demo.

• What if the tool call timed out?
• What if the tool response was malformed?
• What if the model vendor was unavailable?
• What if you seamlessly degraded to a fallback model but it somehow changed the agent's behaviour?
• What if that agent states it's 'done' after that failed tool call, or worse, simply carries on?

Running your agent, and seeing it work, tells you little about how it will break in production.

Traditional software often fails around explicit guarantees (contracts): a parameter was not passed, the return format didn't match the expected shape. Agents are exposed to these same failures but add their interpretation of those errors on top in how they deal with them. Do they continue after a failed tool call, retry it, infer missing context, or pause execution to request clarity - or do they simply report success?

Introducing Chaos

Agent Chaos is the ability to intentionally inject controlled failure modes into an agent's run so you can observe its specific behaviour in response. It allows you, per agent run, to inject controlled, bounded faults and then fully introspect the agent's response.

The point is not to break your agent, it's to build evidence.

We want to be able to demonstrably answer questions like:

• If a critical tool fails, what does the agent do?
• Does the agent tell the user the truth?
• Does the agent accurately present the outcome of the run?
• Does the agent stop before actioning potentially unsafe side effects?
• Is revision B more resilient than revision A?

Agent Chaos is one of the primary tools we're using for resilience testing our agentic workflows.

Embracing Chaos

Chaos runs define controlled scenarios with repeatable injection. The crucial component here is that those faults become both deterministic and observable. When automated using evals, we can then define the acceptable reaction to each fault.

A single chaos-enabled run is interesting. 50 chaos-enabled runs build a body of evidence and confidence in behaviour. Couple this with our ability to build, evaluate and deploy incremental behaviour through versioned agent revisions and the value becomes not just knowing that the agent can work but instead knowing exactly how its behaviour changes across those evaluated scenarios.

Chaos without observability is just noise. For a chaos-enabled run we record: what was injected where, why that fault was injected, what the agent's response to the fault was, and what the result of the run was.

This gives us the ability to make evidence-backed statements like:

The run failed because the tool call was deliberately made to fail. The agent detected that failure, avoided any side effects, explained its limitation, and ended cleanly.

Here's an all-too-common example of bad output given an agent run:

Run Complete

Given some chaos-enabled tuning, here's a much better example:

I could not complete this task because the calendar lookup failed. I did not make any changes. You can retry the task.

Using our platform, the fault-injected agent run above looks like:

curl -N -X POST "${BASE}/task-agent/${TASK_AGENT_ID}/run" \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "message": "List the events in my Google Calendar for tomorrow.",
      "chaos": { "enabled": true, "seed": "calendar-list-tool-failure", "max_faults": 1,
        "modes": [ { "type": "tool_error", "probability": 1, "retryable": false,
          "targets": { "sources": ["integration"], "tool_names": ["google_calendar_list_events"] } } ] }
    }'

The run_id returned in the final streamed event can then be used to inspect the associated TaskAgentRun and its persisted chaos telemetry:

curl -s "${BASE}/task-agent/${TASK_AGENT_ID}/run/${TASK_AGENT_RUN_ID}" \
    -H "Authorization: Bearer ${TOKEN}" \
  | jq '{
      status,
      is_chaos_run,
      chaos_config,
      chaos_events: [
        .chaos_events[] | {
          event_id,
          type,
          applied,
          target,
          phase,
          tool_execution_skipped,
          retryable
        }
      ]
    }'

For the run above, the associated output would show:

{
  "status": "completed",
  "is_chaos_run": true,
  "chaos_config": {
    "enabled": true,
    "seed": "calendar-list-tool-failure",
    "run_index": 0,
    "max_faults": 1,
    "modes": [
      {
        "type": "tool_error",
        "probability": 1,
        "retryable": false,
        "targets": {
          "sources": ["integration"],
          "tool_names": ["google_calendar_list_events"]
        },
        "variant": null
      }
    ]
  },
  "chaos_events": [
    {
      "event_id": "chaos-001",
      "type": "tool_error",
      "applied": true,
      "target": {
        "source": "integration",
        "tool_name": "google_calendar_list_events"
      },
      "phase": "before_tool",
      "tool_execution_skipped": true,
      "retryable": false
    }
  ]
}

Whilst this snapshot shows only the Chaos specific data for this run (we could, of course, also be looking at the agent output, token usage, phase timings, individual events etc) it demonstrates how each run contributes empirical data to a growing corpus of evidence.

Thriving in Chaos

Agent Chaos can never guarantee an Agent is safe, absolutely correct, or even prepared for every failure production may throw at it. It does provide robust tooling to allow for the precise and observable triggering of specific failure modes, removing that guesswork and ultimately enhancing confidence in its overall resilience posture.

Whilst the happy path is always going to be the shiny demo, the various failure modes are where we truly learn about the agent, its capabilities and its overall resilience. This is a foundational pillar of operational maturity.

Our aim at rightbrain.ai is to make resilience a first-class feature of the Agent runtime. We want teams to build, evaluate and deploy Agents, backed with evidence on how they behave when faults arise, so they can focus on solving the real business problems and accelerate their velocity.

References

1. Kubernetes Self-Healing, Kubernetes documentation.
2. Observability, Kubernetes documentation.
3. WAREX: Web Agent Reliability Evaluation on Existing Benchmarks, Microsoft Research.
4. Benchmarking Failures in Tool-Augmented Language Models, Treviño et al., NAACL 2025.
5. Principles of Chaos Engineering.
6. Demystifying evals for AI agents, Anthropic.
7. Testing for Reliability, Google Site Reliability Engineering.