eval-mock

A flight simulator for AI agents. A deterministic, stateful simulated business world where your agent is tested — before it ever touches a real customer, a real order, or a real inbox.

pip install eval-mock
eval-mock quickstart        # first passing verdict in seconds, fully offline

Three pillars

1. A world, not canned responses. One entity store shared by all tools: send an email and the simulated recipient's inbox changes; cancel an order and every later query sees it. Tools are projections of the world.
2. Deterministic by construction. Same scenario + same seed = same world, same journal, same canonical digests — verified byte-identical across 3 OS × 2 Python versions in CI on every commit. Runtime is zero-LLM; sockets and the wall clock are blocked by guards during a run.
3. Verdicts are evidence, not vibes. Every tool call is journaled; assertions run over the final world, the diff, and the journal; failures get earliest-ring attribution; any third party can eval-mock verify your run_dir and reproduce the verdict.

Where to go

• Quickstart — install to first verdict, then your own agent.
• Scenario DSL — one YAML file = one repeatable exam.
• Integrations — MCP, OpenAI, Anthropic, pytest, DeepEval.
• Reproducibility — digests, repro_id, verify.
• Comparison — what we are not, and when not to use us.

Promises

No telemetry, ever. No feature walls, ever — the open-source package is the full product. In writing: PHILOSOPHY.md.