Case study 22 / 37
ChaosProxy-L7
A loopback-by-default proxy that makes HTTP services and PostgreSQL misbehave on purpose, reproducibly and scoped by route, header or query, for resilience tests.
- Status
- Experimental
- Period
- 2026
- Domain
- Cloud · Tools
- Language
- Rust
- Last push
- 26 OCT 2026
- License
- MIT OR Apache-2.0
- Source of claims
- README.md, docs/ARCHITECTURE.md, docs/LIMITATIONS.md, CHANGELOG.md, src/http/mod.rs, src/rules.rs, src/wasm.rs, tests/http_concurrency.rs, tests/http_malformed.rs, .github/workflows/ci.yml
A Rust proxy that speaks HTTP/1.1, h2c and the PostgreSQL v3 wire protocol, so faults can be a 503 on one route, a slow first byte, a TCP reset, or SQLSTATE 40001 on one statement. Percentage rolls come from a seed and the request sequence number, so the same seed and request order give the same faults. Optional WebAssembly matchers run sandboxed.
01/The problem
Timeouts, slow first bytes, dropped connections and serialization failures cause real outages but are hard to trigger on demand. TCP-level tools cannot answer one HTTP route with a 503 or fail one SQL statement with a specific SQLSTATE.
02/The system
A Rust proxy that speaks HTTP/1.1, h2c and the PostgreSQL v3 wire protocol, so faults can be a 503 on one route, a slow first byte, a TCP reset, or SQLSTATE 40001 on one statement. Percentage rolls come from a seed and the request sequence number, so the same seed and request order give the same faults. Optional WebAssembly matchers run sandboxed.
- Client to Listener
- Listener to Service: decide
- Service: decide to Wasm matcher
- Service: decide to Forwarder
- Service: decide to Injected fault
- Forwarder to Upstream
- PostgreSQL session to Service: decide
03/Implementation
- 01HTTP faults: latency with jitter, first-byte delay, bandwidth throttle, TCP reset (SO_LINGER=0), injected 504 timeout, bounded hang and synthetic error responses.
- 02Rules match on path prefix or regex, method, exact headers and an optional WebAssembly module; first match wins, with per-rule concurrency caps.
- 03A PostgreSQL v3 proxy: startup, declined TLS, relayed CancelRequest, SCRAM passthrough, simple and extended query protocol, with error, delay, drop-mid-result and reject-startup faults.
- 04WebAssembly matchers on Wasmtime with no imports, fuel, wall-clock and memory limits and a fresh instance per decision; runtime errors fail closed.
- 05Exactly one JSON access-log line per request, including aborted ones; bodies, header values, query strings and SQL text are never logged by default.
04/Engineering
Decisions are a function of seed and sequence
The decision for request n depends only on the seed, n and the request itself, including jitter, tested with 300 racing requests. The docs spell out exactly what stops being deterministic under concurrency.
Forwarding never decides
The forwarding path never makes fault decisions and the rules layer never touches sockets, which keeps each side testable on its own.
Two opt-ins to leave loopback
Listening beyond loopback needs both a config flag and a CLI flag, and every upstream must be in an allowlist or startup is refused.
05/Interface
No product screenshots are published for this project. The visual above is a code-driven representation of how it behaves, built from the repository source — not a screenshot.
06/Tech stack
- Rust
- Tokio
- hyper
- HTTP/1.1 · h2c
- PostgreSQL wire protocol
- Wasmtime
- TOML
07/Result
Verified outcomes
- Concurrency tests with 512 simultaneous requests across seven fault kinds check fault/log agreement and that no permits, tasks, descriptors or upstream connections leak; malformed-input tests cover CL+TE smuggling and invalid chunked bodies.
- Live PostgreSQL tests on throwaway SCRAM clusters run in CI on Ubuntu; CI on Linux, macOS and Windows; MIT OR Apache-2.0.
Known limitations
- A testing tool only: no TLS on either side, no client authentication; never for production traffic.
- One upstream per listener; WebSocket and CONNECT get 501; no benchmarks, so no performance claims.
- An injected error inside a PostgreSQL transaction does not abort the server-side transaction.
08/Links
Related work
Temporal SQL database
TesseraDB →
An embedded SQL database where the past is a first-class query: every row version retained, AS OF reads, and event streams in the same transactions.
Open source · Research
Optimizing compiler
Bellows →
An optimizing compiler you can read end to end, from source text to x86-64 machine code, with an IR interpreter that must agree with the native binary on every program.
Open source · Research
Programmable HTTP/1.1 server
Spindrift →
An HTTP/1.1 server with its own RFC 9112 protocol stack, a pipeline language instead of a config file, and reloads that never drop a request.
Open source · Research
Next case study
AST-StateFuzzer →