Bare-metal benchmark results

CI numbers in latest.md come from a 2 vCPU shared GitHub Actions runner. They are useful for catching order-of-magnitude regressions; they are not representative of the hardware weir is designed to run on, and they are not the numbers any performance claim should be made against.

This page holds the numbers captured on a real machine with named hardware. It is the ship gate, and as of 2026-09-18 it is an operable one: the capture below is the first this file has ever held, and the run-to-run spread under it is measured rather than assumed, so its thresholds can be checked against the noise instead of hoped to exceed it.

Re-capture with deploy/run_bare_metal_bench.sh — see Capture procedure, which is not a whole-file overwrite.

Latest capture

Captured: 2026-09-18 21:36:54 UTC Host: beast CPU: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz — 16 logical cores, microcode 0xf8 Memory: 32023 MiB Kernel: Linux 7.0.0-31-generic x86_64 libc: glibc 2.39

Storage (load-suite wab dirs land in /tmp): Filesystem: ext4 Mount opts: rw,relatime Block devices:

NAME    MODEL                     ROTA TRAN
sda     Samsung SSD 860 EVO 1TB      0 sata
sdb     ST4000DM006-2G5107           1 sata
sdc     Samsung SSD 850 EVO 250GB    0 sata
nvme0n1 Samsung SSD 970 EVO 500GB    0 nvme

Tunables: Governor: schedutil SMT: on Turbo: CPU mitigations: off vm.dirty_background_bytes: 0 vm.dirty_bytes: 0 vm.dirty_expire_centisecs: 3000

Run config: 5 passes × 2 deadlines (1 2 ms)

Version: 4.0.0
Last updated: 2026-09-18 21:36 UTC
Averaged over: 5 run(s) per deadline
Server config: shard_count=4, batch_size=64

Operator-run capture on named hardware. The environment header above records the machine, its storage and its tunables; two captures are comparable only if those match. This is the surface environments.md permits an external claim to cite — unlike latest.md, which is a shared-runner regression detector.

Throughput — deadline comparison

Latency — single thread, batch_deadline_ms=1

Latency — single thread, batch_deadline_ms=2

Saturation Ramp — Buffered tier

Server started with max_connections = 48. Levels above 48 threads trigger connection-cap exhaustion; the server must survive every level.

Saturation Ramp — Sync tier

Server started with max_connections = 48. Uses Sync durability to stress the group-fsync path under escalating concurrency.

Generated by deploy/avg_benchmarks.py

Run-to-run noise floor

A threshold below the noise floor blocks releases on nothing, so before these gates were allowed to bind, the same code was captured four times back to back on an otherwise idle beast (2026-09-18, weir 4.0.0). The spread across those four captures is the smallest difference this surface can actually resolve.

Only one of the two ungateable rows was already known to be noisy. tests/load.rs says the thundering_herd_* scenarios are warmup-dominated, swing run to run, and should be read as a smoke test rather than a throughput gate, preferring the ramps as the stable baseline; the 11.0% measured here agrees with that warning and puts a number on it.

connection_churn carries no such warning, and its 23.3% is a new result. It is the widest spread of any scenario in the suite — wider than the herd scenarios its own file singles out — and it is the only scenario whose timed work is dominated by repeated connect/disconnect rather than by pushing records. Nothing here establishes why; what it establishes is that the number cannot carry a 10% gate, and that load.rs's advice about which scenarios are stable is incomplete.

Latency and the ramps are steadier than the throughput rows. Sync p99 read 3.7 ms in every one of the four captures, so the 20% gate has at least an order of magnitude of headroom at the precision published. Both saturation ramps reproduced exactly: ok through 48 threads, SATURATED at 64 with 16 dropped I/Os, and 48 dropped at 96 — identical counts in all four. A status regression there is signal, not noise.

This is what the CI surface cannot do. history.md spans 1,926–2,783 single-thread Sync RPS (1.45x) for one released version with no code change; beast spans 838–853 (1.018x) — the same measurement, ~25x tighter. Note also which way the absolute numbers go: CI reports two to three times beast's Sync throughput, on far weaker hardware, because a virtualised fsync is cheap and beast's fdatasync to a SATA SSD is honest. That is the single clearest argument for the citation rule in environments.md.

Regression policy

These thresholds are now validated for the metrics they name — see the spread above. They are not valid for connection_churn or for thundering_herd_64_threads at a 2 ms deadline, whose own run-to-run ranges (23.3% and 11.0%) exceed the 10% threshold; a gate on either fires on noise. Compare those two by direction across several captures, or not at all.

Re-validate the floor whenever the bench machine changes, the load suite's scenarios or sample counts change, or the capture starts landing on different storage. The spread is a property of the machine and the suite together, not a constant.

The CI gate exists to catch the kind of mistake a code review wouldn't (a missing #[inline], an accidental Mutex on the hot path, a serialisation step that quietly went from zero-copy to allocating). Anything smaller than an order of magnitude is below CI's noise floor and has to be re-tested against bare-metal numbers before it counts.

Capture procedure

The script captures every piece of context needed to compare two runs:

• CPU model, core count, microcode revision, current governor
• Kernel version, libc version, glibc tunables
• Filesystem type and mount options for wab_dir
• Block device model (lsblk -d -o NAME,MODEL,ROTA,TRAN)
• vm.dirty_background_bytes, vm.dirty_bytes, vm.dirty_expire_centisecs
• Whether mitigations=off, SMT, and turbo are enabled

It then runs the load suite 5× at each of batch_deadline_ms ∈ {1, 2} (same as CI), feeds the JSONL through avg_benchmarks.py, and writes the combined env-header + result tables to stdout.

# On the target machine, after a clean build:
deploy/run_bare_metal_bench.sh > /tmp/weir-capture.md

Then replace the region between the BEGIN CAPTURE and END CAPTURE markers above with that file's contents, minus its # Bare-metal benchmark results heading, and commit.

Do not redirect the script over this file. Earlier revisions documented run_bare_metal_bench.sh > docs/benchmarks/bare-metal.md, which replaces the whole document — deleting the regression policy, the measured noise floor, this procedure and the environment annotations below, all of which are what makes the capture interpretable. The script emits a capture, not this page.

Re-capture after any of:

• CPU / kernel / libc upgrade on the bench machine
• A change to weir-core, weir-server, or the WAB on-disk format
• A change to the load suite's scenarios or sample counts

Environment annotations

Every captured bare-metal.md must carry a header that names:

Captured: <UTC timestamp>
Host: <hostname>
CPU: <vendor, model, MHz, cores/threads, microcode>
Memory: <total MiB, type if known>
Kernel: <uname -r>
Storage (wab_dir):
  Path: <absolute path>
  Filesystem: <type, mount options>
  Device: <model, rotational/SSD, NVMe/SATA>
Tunables:
  Governor: <performance | schedutil | …>
  SMT: <on | off>
  Turbo: <on | off>
  vm.dirty_background_bytes: <value>
  vm.dirty_bytes: <value>

A bare-metal run without this header is not a bare-metal result.

Two captures are comparable only if the storage rows match. Durable-tier throughput is bounded by one fdatasync per record on a single connection — the daemon acks frame N before reading frame N+1 (crates/weir-server/src/socket/connection.rs:485-491) — so the number reports the platform's fsync primitive as much as it reports weir. Measured on one M3 Max, the three primitives this codebase can reach cost 36 µs (plain fsync(2), which is what File::sync_all is on Linux), 238 µs (F_BARRIERFSYNC, what the WAB record path uses on macOS — crates/weir-server/src/wab/segment.rs:672-685) and 3,965 µs (F_FULLFSYNC, what File::sync_all becomes on macOS, and what a segment seal's parent-directory fsync and the drain's confirm path both pay). run_bare_metal_bench.sh runs on macOS as well as Linux, so this is a real hazard: a capture on a different OS or filesystem is a different measurement, not a regression.