weir Wire Protocol — Conformance Vectors

This directory ships a language-neutral conformance suite for the weir v1 wire protocol: conformance/wire_v1_vectors.json. If you are writing a weir client or daemon in another language, run your codec against these vectors to confirm it is byte-compatible with the reference implementation. The format itself is specified in wire_protocol.md; this file describes only the vectors.

What's in the file

{
  "wire_version": 1,
  "max_payload_hard_cap": 16777216,
  "description": "...",
  "vectors": [ { ... }, ... ]
}

Each entry in vectors is one test case:

The two kinds of vector

• decode == "ok" — hex is exactly one valid frame. A conformant decoder must accept it and produce the stated message_type, durability, flags, and payload. Encoding those fields back must reproduce hex byte-for-byte (encode/decode are inverses) — unless the vector sets "round_trip": false, which marks a vector that decodes a non-canonical wire value (only push_batched, the retired 0x02 durability byte, today) to its canonical field value. Such a vector's hex is still frozen and must still decode correctly; it is simply exempt from the byte-identical re-encode check, since canonicalisation is one-way by design.

• decode == "<Error>" — hex must be rejected, and the error must be the named one. The rejection tags are:

  Tag	Cause
  BadMagic	First four bytes are not WEIR.
  VersionMismatch	Version byte ≠ 1.
  UnknownMessageType	message_type byte has no known variant.
  UnknownDurability	durability byte has no known variant.
  HeaderCrcMismatch	Header CRC does not cover bytes [0..12].
  PayloadCrcMismatch	Trailing CRC does not match the payload.
  TruncatedFrame	Buffer shorter than 16 + payload_len + 4.
  PayloadTooLarge	Declared payload_len exceeds max_payload_hard_cap.
  ReservedFlagsSet	flags byte is nonzero.
  TrailingBytes	Buffer longer than the single frame it declares (G18).

Your decoder need not use these exact names internally — only map each vector's tag to your equivalent rejection. The ordering guarantees that make some of these deterministic (e.g. version is checked before the header CRC, the payload cap before the frame-length check) are documented in wire_protocol.md; the vectors encode the observable result, not the internal order.

Decoder tag → wire Nack byte

The rejection tags above are decoder names (they describe what the codec detected). They are not the same vocabulary as the wire NackReason bytes the daemon sends back — several decoder tags collapse to one Nack byte, and two are decode-only with no Nack at all. When a Push fails decoding, the daemon maps the decoder verdict to a wire Nack as follows:

EmptyPayload (0x07) and InternalError (0x06) are wire Nacks with no decoder tag: an empty Push is a daemon admission-policy rejection (the codec accepts a zero-length payload), and InternalError is a runtime/transient condition, not a decode verdict. The full byte table is in wire_protocol.md; the source of truth for this mapping is nack_for_decode_error in crates/weir-server/src/socket/connection.rs.

Coverage

Known gap: BadBatchFraming (0x0B) has no vector. It is an assigned, wire-emitted reason — the one a client gets for most malformed PushBatch frames — and it appears in none of the vector files. The frozen wire_v1_vectors.json cannot gain it (the freeze forbids changing that file), so it belongs in wire_v1_batch_vectors.json. Until then, the counts below describe the vectors that exist, not the full assigned reason set.

The suite covers every message type, the nine Nack reason bytes 0x01–0x09 as decodable Nack frames, and at least one rejection vector per DecodeError variant, including the boundary cases (empty payloads, the payload-cap boundary, a truncated header vs. a truncated payload, trailing bytes after a complete frame, and a doubly-malformed header that pins the field-parse precedence — unknown durability and a nonzero flags byte resolves to UnknownDurability, since durability is parsed before the flags check).

The tracked-push extension — a second, separate file

PushTracked (0x06) and AckTracked (0x07) have their own vector file, conformance/wire_v1_tracked_vectors.json, generated by conformance/gen_tracked_vectors.py.

wire_v1_vectors.json is untouched by that extension, and always will be. A decoder that implements only message types 0x01–0x05 is correct to reject a 0x06 frame as UnknownMessageType — that rejection IS the protocol's version negotiation. Putting a tracked vector in the frozen file would therefore make every conformant v1 client fail a vector it is right about. So:

• If you do not implement tracked pushes, ignore this file entirely. You are still fully conformant.
• If you do, run it in addition. run_vectors.py checks every file and reports them on separate lines.

The tracked file has two vector lists rather than one:

A coordinate_vectors entry decodes to segment, index and record_id_hex, or names one of these rejection tags:

The record_id_hex in these vectors is generated with Python's hashlib SHA-256 and verified against the Rust sha2 crate, so a passing vector means two independent SHA-256 implementations agree — the same cross-check the CRC vectors give for crc32fast. It is the value the drain hands a sink as its per-record idempotency key, so a client that reproduces it can correlate its pushes with what the downstream received.

Regenerate with:

python3 docs/conformance/gen_tracked_vectors.py > docs/conformance/wire_v1_tracked_vectors.json
cargo test -p weir-core --test conformance_tracked

The batch extension — a third, separate file

PushBatch (0x08) and AckBatch (0x09) have their own vector file, conformance/wire_v1_batch_vectors.json, generated by conformance/gen_batch_vectors.py.

The reasoning is exactly the one above, applied again: a decoder implementing only 0x01–0x05 is correct to reject an 0x08 frame, so a batch vector in the frozen file would fail every conformant v1 client. A guard in crates/weir-core/tests/conformance.rs asserts the frozen file carries no type byte above 0x05 at all — stated as a range rather than as a list of the extensions that exist today, so the next one is covered the day its byte is assigned rather than the day someone remembers to widen a list.

• If you do not implement batching, ignore this file entirely. You are still fully conformant.
• If you do, run it in addition. run_vectors.py checks all three files and reports them on separate lines.

The batch file has three vector lists:

An ack_vectors entry carries expected because an AckBatch cannot be decoded without it: the bitmap's length is ceil(N/8), which is not injective, so the count the client sent is the only thing that can catch a desync. An "ok" entry gives its verdicts as accepted (a list of booleans) or, when they are all the same, as accepted_all — the at-cap vector would otherwise carry 2048 booleans.

Both lists share one rejection-tag vocabulary:

The vector this file exists for

ack_bitmap_asymmetric_n9 is the one that earns the whole file. An AckBatch written with the opposite bit order is a well-formed frame — header CRC valid, payload CRC valid, payload_len correct — that reports failures as successes, and nothing in the protocol detects it.

Every vector with N ≤ 8, and every symmetric pattern at any N, encodes identically under both conventions and would pass against an inverted decoder. N = 9 with records 0, 2 and 8 accepted encodes as 05 01 under weir's LSB-first convention and a0 80 under MSB-first: N > 8 forces the byte order to show, the asymmetric pattern forces the bit order to show, and neither alone is enough.

Two of the rejection vectors carry no record data at all (reject_body_too_many_records, reject_body_record_over_the_payload_cap). They are deliberately tiny, because what they pin is the order of the checks: a declared count and a declared record length must each be rejected against the cap before anything is sized by them. An implementation that reserves first, or that tests bounds before the cap, fails these and passes a version that carries the data.

Regenerate with:

python3 docs/conformance/gen_batch_vectors.py > docs/conformance/wire_v1_batch_vectors.json
cargo test -p weir-core --test conformance_batch

CRC algorithm

Both CRC fields are IEEE / ISO-3309 CRC-32 — the zlib/PNG/Ethernet variant, not CRC-32C (Castagnoli). The vectors are generated with Python's zlib.crc32 and verified against the Rust crc32fast crate, so a passing vector means two independent CRC implementations agree on the bytes. The same variant is exposed by Go's hash/crc32.IEEETable, Java's java.util.zip.CRC32, and Node's node:zlib.crc32 (Node >= 22.2). See wire_protocol.md for the full parameter table.

Running the suite

Rust (reference implementation):

cargo test -p weir-core --test conformance

crates/weir-core/tests/conformance.rs loads this JSON, decodes every vector with weir-core, and checks the result — and, for "ok" vectors, that re-encoding round-trips to the same bytes. Each extension has its own suite — cargo test -p weir-core --test conformance_tracked and --test conformance_batch. The guard that no extension frame reaches the frozen v1 file lives with the frozen set in conformance.rs, not in either extension's suite: the claim is a property of that file, and it must survive an extension being removed.

Other languages: a ready-to-run reference harness ships at conformance/run_vectors.py (stdlib-only Python). It includes a small reference codec that passes all vectors, so it doubles as a worked non-Rust implementation:

python3 docs/conformance/run_vectors.py   # "30/30 vectors passed", plus a line per extension;
                                          # non-zero exit on any mismatch

It also checks wire_v1_tracked_vectors.json and wire_v1_batch_vectors.json when those files are present, reporting each on its own line — including, for both, a check that every extension frame is rejected as UnknownMessageType by the v1-only decoder, which is the compatibility claim made executable. Its batch codec is written out in full rather than ported, so a passing run means two independent implementations agree on the bitmap convention.

To validate your own client, replace its decode_frame() / encode_frame() with adapters over your codec and run it unchanged. (Or, from scratch: load the JSON, hex-decode each hex, run your decoder, and assert the outcome matches decode — and the header/payload fields when "ok".)

Regenerating the vectors (maintainers)

The vectors are generated, never hand-edited, by conformance/gen_vectors.py:

python3 docs/conformance/gen_vectors.py > docs/conformance/wire_v1_vectors.json
cargo test -p weir-core --test conformance   # confirm weir still agrees

After an intentional wire change, regenerate the file and re-run the Rust suite; the test deliberately fails if the bytes drift so the change is never silent. A breaking change to any "ok" vector requires a WIRE_VERSION bump.