Multi-agent testing and deployment

Reproducible worlds for AI agents

Shakedown runs your agent teams against stateful copies of the systems they touch, scores the end state with deterministic checks, and replays each step. Control runs the passing teams in production, in any industry.

How we test

How a test run works

  1. 01 · Compose

    Compose a world

    Each system joins through a connector: an ERP, a plant, a database, a simulated supplier hub. Each run gets fresh copies.

  2. 02 · Save

    Snapshot the world

    Databases, files, queues and the world clock save at one step. Systems without exact saves replay from a seed.

  3. 03 · Inject

    Inject faults

    The scenario schedules faults on the world clock: a machine alarm at 7:40, a supplier delay at 9:15, a rush order at 1:30.

  4. 04 · Run

    Run setups, interleaved

    Up to four agent setups, N runs each, interleaved so no setup gets easier conditions. Limits hold in the environment, and denied calls count.

  5. 05 · Score

    Score the end state

    Deterministic goal and safety checks read the final state. Each pass rate ships with a 95% Wilson interval.

Architecture →
Shakedown · Press 4 failure06:00:00
ERPorders · stock · BOMs
PlantIsaac Sim · OPC UA
CRMcustomers · promises
Suppliersreply on schedule
Email + paymentssealed · never sent
Quality · LIMSflags · inspections
WorldStamping plant6 systems
Six systems, one test
Save point06:00:006 of 6 saved
Run 1 · restoredOrders and stock as saved
Run 2 · restoredClock back at 06:00
Run 3 · restoredInboxes emptied
Run 20 · restoredIdentical starting state
The snapshot behind twenty runs
06:0009:0012:0015:0018:00
07:40 · PlantPress 4 hydraulic fault, line stops
09:15 · SupplierCastings slip three days
11:00 · MESA shop-floor confirmation arrives late
13:30 · CRMRush order from the largest customer
Each problem hits at a set time
Setup A
One agent · general rules
Run 1/10
Setup B
Planner + buyer · site rules
Run 1/10
Ten runs per setup
Setup AOne agent · general rules
0%6 of 10 passed
95% interval31–83%
Flagged material used0
Denied calls0
Model cost per run$0.41
Setup BPlanner + buyer · site rules
0%9 of 10 passed
95% interval60–98%
Flagged material used0
Denied calls0
Model cost per run$0.68
Sample experiment · synthetic data

Engine

What the engine does

Restore

Save, restore, fork

Each connector declares exact save or replay. Branch any run at any step.

Checks

Deterministic scoring

Goal and safety checks run as code against the end state.

Experiments

Setups side by side

Up to four setups, interleaved runs, pass rates with a 95% interval.

Limits

Enforced in the environment

Purchasing caps and approval gates hold outside the agent. Denied calls count.

Architecture →

The platform

Three products, one engine

Environments composes worlds from connectors. Shakedown runs experiments and suites in those worlds. Control runs agents in production with the same event format.

Get started

Run your agent in a world

One workflow, one world, a few hundred runs. You leave with traces, intervals and a suite to rerun on each change.