Benchmarks
The promoted public matrix is 20260810T105308Z-f3fd294. It ran on the project's Oracle Ampere A1 ARM64 host and contains 51 rows each for Lockwell, MinIO, Garage, and SeaweedFS, zero request errors, and five completed repair/scrub/backup/restore drills. Its generated competitive gate has 20 failures, so the promoted evidence does not support a performance-leadership or provider-replacement claim. The immutable manifest, hashes, raw rows, profile, and gate are in benchmark-baselines/phase1/.
The latest complete unchanged-policy diagnostic is bench-results/20260815T-full-access-log-23194cf, bound to merged 23194cf2dcbc93bad904d1743d82c365f9b4fe4d; it reduced the generated gate to 10 failed checks out of 142 with the same four targets and five drills, but it is not committed or promoted. Until a reviewed run is promoted, the tracked baseline above remains the public release ledger and B-001 remains active.
The explorer below is the historical Lockwell-vs-MinIO presentation dataset. It remains useful for navigating operation, size, concurrency, throughput, and latency dimensions, but it is not the current four-target release ledger. Read the caveats before quoting anything; they are part of the result.
How to read latency Throughput (MiB/s, ops/s) is "how much per second": higher is better. The p50 and p95
views are response times in milliseconds: p50 is the median request, p95 the slow tail, and lower is better. A Lockwell p50 at half of MinIO's means Lockwell answers twice as fast. The Advantage column already does this arithmetic for you, in the right direction, on every view. :::
The ledger
higher is better · advantage column = how many times better Lockwell is on this metric, in either direction
| Operation | Size | Clients | Lockwell | MinIO | Advantage |
|---|---|---|---|---|---|
| put | 4 KiB | 1 | 3.20 MiB/s | 0.56 MiB/s | 5.7x |
| put | 4 KiB | 16 | 17.4 MiB/s | 4.12 MiB/s | 4.2x |
| put | 4 KiB | 64 | 10.5 MiB/s | 9.09 MiB/s | 1.2x |
| put | 1 MiB | 1 | 155.4 MiB/s | 38.4 MiB/s | 4.0x |
| put | 1 MiB | 16 | 1,381 MiB/s | 188.8 MiB/s | 7.3x |
| put | 1 MiB | 64 | 1,622 MiB/s | 373.5 MiB/s | 4.3x |
| put | 64 MiB | 1 | 308.1 MiB/s | 266.0 MiB/s | 1.2x |
| put | 64 MiB | 16 | 1,593 MiB/s | 598.7 MiB/s | 2.7x |
| put | 64 MiB | 64 | 1,631 MiB/s | 400.9 MiB/s | 4.1x |
| get | 4 KiB | 1 | 11.3 MiB/s | 4.53 MiB/s | 2.5x |
| get | 4 KiB | 16 | 47.3 MiB/s | 45.7 MiB/s | 1.0x |
| get | 4 KiB | 64 | 78.5 MiB/s | 48.0 MiB/s | 1.6x |
| get | 1 MiB | 1 | 944.5 MiB/s | 256.9 MiB/s | 3.7x |
| get | 1 MiB | 16 | 8,598 MiB/s | 3,869 MiB/s | 2.2x |
| get | 1 MiB | 64 | 8,544 MiB/s | 5,203 MiB/s | 1.6x |
| get | 64 MiB | 1 | 2,858 MiB/s | 2,460 MiB/s | 1.2x |
| get | 64 MiB | 16 | 10,889 MiB/s | 4,733 MiB/s | 2.3x |
| get | 64 MiB | 64 | 5,410 MiB/s | 3,524 MiB/s | 1.5x |
| head | 4 KiB | 1 | 2,510 ops/s | 2,298 ops/s | 1.1x |
| head | 4 KiB | 16 | 11,512 ops/s | 13,179 ops/s | 0.9x behind |
| head | 4 KiB | 64 | 23,451 ops/s | 18,741 ops/s | 1.3x |
| head | 1 MiB | 1 | 3,123 ops/s | 1,007 ops/s | 3.1x |
| head | 1 MiB | 16 | 18,111 ops/s | 20,576 ops/s | 0.9x behind |
| head | 1 MiB | 64 | 42,878 ops/s | 43,716 ops/s | 1.0x behind |
| head | 64 MiB | 1 | 3,149 ops/s | 2,208 ops/s | 1.4x |
| head | 64 MiB | 16 | 17,290 ops/s | 20,865 ops/s | 0.8x behind |
| head | 64 MiB | 64 | 52,432 ops/s | 14,688 ops/s | 3.6x |
| list | 4 KiB | 1 | 2,001 ops/s | 524.3 ops/s | 3.8x |
| list | 4 KiB | 16 | 4,095 ops/s | 1,848 ops/s | 2.2x |
| list | 4 KiB | 64 | 2,737 ops/s | 464.9 ops/s | 5.9x |
| list | 1 MiB | 1 | 2,076 ops/s | 1,351 ops/s | 1.5x |
| list | 1 MiB | 16 | 5,114 ops/s | 2,044 ops/s | 2.5x |
| list | 1 MiB | 64 | 2,699 ops/s | 564.5 ops/s | 4.8x |
| list | 64 MiB | 1 | 2,067 ops/s | 1,687 ops/s | 1.2x |
| list | 64 MiB | 16 | 7,024 ops/s | 1,818 ops/s | 3.9x |
| list | 64 MiB | 64 | 3,661 ops/s | 503.9 ops/s | 7.3x |
| multipart-put | 64 MiB | 1 | 222.5 MiB/s | 232.6 MiB/s | 1.0x behind |
| multipart-put | 64 MiB | 16 | 985.1 MiB/s | 413.4 MiB/s | 2.4x |
| multipart-put | 64 MiB | 64 | 1,036 MiB/s | 532.9 MiB/s | 1.9x |
| multipart-copy | 64 MiB | 1 | 305.4 MiB/s | 205.4 MiB/s | 1.5x |
| multipart-copy | 64 MiB | 16 | 3,179 MiB/s | 325.3 MiB/s | 9.8x |
| multipart-copy | 64 MiB | 64 | 2,300 MiB/s | 338.9 MiB/s | 6.8x |
| mixed-rw | 4 KiB | 1 | 1,714 ops/s | 338.0 ops/s | 5.1x |
| mixed-rw | 4 KiB | 16 | 6,549 ops/s | 2,868 ops/s | 2.3x |
| mixed-rw | 4 KiB | 64 | 7,259 ops/s | 1,878 ops/s | 3.9x |
| mixed-rw | 1 MiB | 1 | 316.6 ops/s | 99.9 ops/s | 3.2x |
| mixed-rw | 1 MiB | 16 | 3,279 ops/s | 918.6 ops/s | 3.6x |
| mixed-rw | 1 MiB | 64 | 4,074 ops/s | 1,537 ops/s | 2.7x |
| mixed-rw | 64 MiB | 1 | 10.2 ops/s | 9.29 ops/s | 1.1x |
| mixed-rw | 64 MiB | 16 | 92.3 ops/s | 54.3 ops/s | 1.7x |
| mixed-rw | 64 MiB | 64 | 81.8 ops/s | 27.9 ops/s | 2.9x |
on-disk footprint for the identical write set: Lockwell 44.4 GiB vs MinIO 80.4 GiB (0.55x, lower is better)
2026-06-12 · Windows-11-10.0.26200-SP0 · zero errors required in every row (this run: 0) · reproduce: make bench
Caveats
These are part of the result, not footnotes to hide.
- Durability tier. The bench configuration runs Lockwell in its grouped-durability tier (the write-ahead log is fsynced every 10 ms, not per commit, matching Garage's model; a power loss can cost up to ~10 ms of acknowledged writes). MinIO runs its defaults, which sync per operation. Lockwell's default tier is strict per-commit sync; if your threat model requires it, benchmark that tier instead. This asymmetry flatters Lockwell most on small-object PUT, which is exactly where the warp gap is largest.
- Current host. The promoted four-target baseline ran server and client containers on one Oracle Ampere A1 ARM64 host (four cores, about 24 GiB RAM). Absolute results are host- and image-specific; the complete profile ships so the matrix can be rerun rather than generalized to unrelated hardware.
- MinIO version. Each run pulls
minio/minio:latestat run time. Version-to-version variance is real, so cross-run comparisons of old tables mix that in. - CPU. Lockwell sustains the higher throughput while using more CPU than MinIO at peak. It trades compute for throughput and disk; if you are CPU-bound, weigh that.
- Failed rows remain visible. The promoted baseline records 20 local-leadership failures across GET, HEAD, LIST, mixed-RW, multipart PUT, and PUT; the latest complete diagnostic records 10.
BLOCKERS.mdlists the exact tuples, and the generated gate remains authoritative for each artifact. Neither result clears B-001. - Storage. Core throughput runs disable compression, deduplication, and encryption for an apples-to-apples engine comparison. Feature-profile storage-efficiency results are separate and must not be substituted for the core matrix.
Neutral hardware
Dev-machine numbers carry dev-machine noise. The standing plan is to run the same two harnesses on a fresh low-cost cloud box (the EUR 5 Hetzner class Lockwell is designed to fit), where nothing else is running, the exact specs are public, and anyone can rent the identical machine and check. scripts/bench-remote-hetzner.sh provisions the server with hcloud, runs make bench and make bench-warp, copies the evidence back, and destroys the box; the dataset selector above grows a new entry whenever such a run lands.
Reproduce
make bench # the full matrix harness (Lockwell, MinIO, Garage, SeaweedFS)
make bench-warp # MinIO's warp against Lockwell and MinIO on the same stackBoth write raw per-run evidence under ignored bench-results/. Promote a completed matrix with go run scripts/promote-benchmark-baseline.go ...; only its validated allowlist is committed under benchmark-baselines/phase1/. The historical explorer is regenerated with node website/scripts/build-bench-data.mjs <matrix-dir> <warp-dir>. The methodology and regression thresholds live in docs/benchmark-baselines.md.