NightWatch
{Tech: Europe} Agentic AI Hack × Google DeepMind · team project
Sits in front of a live service. When something breaks, it finds the cause in the code, races three fixes in parallel sandboxes and ships only the one proven to work.
01 · Try it
checkout · live service
healthy
sandbox A · minimal
Idempotency key on capture
+6 −1
- replay the fault
- normal checkout
sandbox B · defensive
Never retry after a timeout
+3 −9
- replay the fault
- normal checkout
sandbox C · structural
Ask the ledger before retrying
+41 −12
- replay the fault
- normal checkout
Incident report
Cause. A dropped connection after payment capture triggered a blind retry, which captured again.
Fix. Sandbox A: an idempotency key on capture. It blocked the double charge, kept checkout working and had the smallest diff. B stopped the charge but left orders stuck as pending. C worked but changed 53 lines.
Held for a human. The patch touches payments, so the safety gate asked before shipping. Once approved, it went live under a reversible lease.
An illustrated replay of the pipeline’s stages on NightWatch’s own demo scenario, with timings compressed. The real run, with Gemini, Modal sandboxes and Jev driving a browser, is in the demo video.
02 · The problem
Most AI incident tools do one of two things: raise an alert, or generate a single fix and hope. The first still wakes an engineer at 3am. The second ships a guess to production.
NightWatch generates several genuinely different fixes, tests each one by driving a real browser through both the attack and a normal user journey, and ships only the fix that blocks the problem without breaking anything else.
03 · How it works
Scroll the steps. The scene follows.
GET/products/sku-a200
POST/cart200
GET/checkout200
POST/checkout/pay504
POST/checkout/payretry
GET/orders/1043200
flagged in under 100 ms
ledger: 2 captures, 1 order
recent deploy? no → diagnose
rule check + Jev on every request · Gemini only when something's wrong
patch Aminimal
+ key = idempotency_key(order) + if ledger.seen(key): return capture(order)
+6 −1
patch Bdefensive
- retry_on_timeout(capture) + capture(order) + mark_pending(order)
+3 −9
patch Cstructural
+ class LedgerInquiry: + def status(self, order): + …
+41 −12
three different fixes, each a validated schema, never a guess
sandbox A
replay the fault
normal checkout
sandbox B
replay the fault
normal checkout
sandbox C
replay the fault
normal checkout
Jev drives a real browser through the attack and a normal journey in each
safety gate
patch A touches payments
held for a human
✓ approved
reversible lease
roll back any time · incident report written
Illustrated from the project’s README and code. Values shown are examples.
- 01
Fast detection, careful repair
A rule check and an inline Jev classifier flag suspicious traffic in under 100 ms. A regression from a recent deploy takes the instant-rollback path. Anything else goes to Gemini, which explores the codebase and logs to find the root cause.
- 02
Three fixes, three sandboxes
Gemini writes three patch candidates as validated Pydantic objects: a minimal diff, a defensive fix and a structural fix. Each boots in its own Modal sandbox with a headless browser attached.
- 03
Proven, not predicted
In every sandbox, Jev replays the exploit and runs a legitimate user flow. A candidate wins only if the exploit is blocked and the flow still works, with the smallest diff breaking ties. If none pass, the service is quarantined and a human is called.
- 04
A deterministic safety gate
Before anything ships, a rule check holds any patch touching auth, payments or data deletion for a human. The winning patch is activated under a reversible lease, and engineers wake up to an incident report instead of an outage.
04 · Architecture
The whole system, running.
Built with