Snapshots — Loading, Verification, and SnapStream¶

Docs-site note (2026-07-11): Part of the planned Vexor documentation site, modeled on the structure and feel of docs.anza.xyz / anza.xyz. Vexor is testnet-only today (0.9.x pre-production); the site will be built out further when Vexor is production-ready. This page is deliberately honest about what is done versus partial: snapshot loading is proven and live, snapshot creation is a partial scaffold, and SnapStream is a design plus proven proofs-of-concept — not yet a shipped production feature. Style: clean headings, plain language, claims grounded in source + measurement.

Overview¶

A Solana validator boots by loading a snapshot — a point-in-time image of the entire account state (every account's lamports, owner, data, and flags) at some recent rooted slot, plus a manifest of bank fields. Without a snapshot a node would have to replay the chain from genesis, which is infeasible.

Vexor's snapshot story has three parts, at three different maturity levels:

Capability	Status	One-line
Loading Agave-format snapshots	Done, live, proven	mmap + parallel parser boots ~86.7M accounts as the validator's account state.
Verification of the loaded base	Done, live	full-snapshot lt-hash guard: `BLAKE3(accounts_lt_hash)` must equal the archive's authoritative checksum.
Creation of snapshots	Partial (~30%)	AppendVec writer + shell-tar packaging exist; in-process zstd encoder and several manifest fields are missing.
SnapStream cross-client restart autopilot	Design + PoC	a planned differentiator; the distributor ships, the restart-coordination autopilot is designed with proven PoCs.

The sections below take each in turn and are explicit about the boundary between what runs in the deployed validator today and what is roadmap.

Snapshot loading (done — proven by boot)¶

Loading is the mature half of the subsystem. The Vexor validator boots from a standard Agave-format snapshot — it reads the same .tar.zst archive layout Agave produces (an accounts/ directory of AppendVec files plus a serialized bank manifest), so it can bootstrap from any snapshot the cluster or a snapshot provider serves.

This is proven by the fact that the node boots and votes bank-exact: if the account state were loaded incorrectly, the very first frozen bank hash would diverge from the cluster. It does not — the node loads ~86.7M accounts and proceeds to vote in line with the cluster.

How it loads — mmap + parallel parse¶

The loader (src/vex_store/parallel_snapshot.zig) is built for fast, low-overhead bootstrap:

Memory-mapped reads. Each AppendVec file is mmap'd read-only (mmapAndIndex, std.posix.mmap with PROT.READ), so account bytes are read zero-copy straight out of the page cache — the loader indexes into the mapping rather than copying every account into a separate buffer. The mapping is advised MADV_SEQUENTIAL for the forward load scan and MADV_HUGEPAGE (the map stays alive for the validator's lifetime and becomes the hot random-access account store after boot).
Parallel AppendVec parsing. The file list is split across worker threads — std.Thread.spawn over getCpuCount() - 1 workers by default — and each thread parses its share of AppendVec files independently with no shared mutable state. Parsing is the CPU-bound step, so spreading it across cores is where the speedup comes from.
Agave on-disk record layout. The parser walks Agave's exact AppendVec record format: StoredMeta (8-byte write_version + 8-byte data_len + 32-byte pubkey = 48 B), then AccountMeta (8 lamports + 8 rent_epoch + 32 owner + 1 executable + 7 padding = 56 B), then a 32-byte hash that sits between the meta and the account data, then the data itself (8-byte aligned). The known Agave quirks are handled: write_version is obsolete (always 0 in modern Agave) and data_len = 0 is valid (e.g. empty wallet accounts).

Measured boot is roughly ~3.8 s to load ~86.7M accounts on the production host. 86.7M accounts is the corroborated working-set size on the live testnet node.

Honest note on parallelism. The loader's source header mentions io_uring batch reads and a "4–8×" speedup target. In the deployed code io_uring is not wired — its shim reports isAvailable() = false and the loader falls back to the mmap + thread-per-core path described above. The real, shipping mechanism is mmap + parallel CPU parsing, which is what boots the node; io_uring is a future optimization, not a current claim. The "4–8×" / "30–40% faster" figures in the source are aspirational targets, not measured results.

Snapshot verification (done — live)¶

Loading the right bytes is not enough; the validator must also verify the loaded base is the authentic state for that slot before it trusts it. Vexor does this with a full-snapshot lattice-hash (lt-hash) guard that runs unconditionally at boot (src/vex_svm/bootstrap.zig, verifyBaseLtHashAgainstArchive).

The check, in Agave's terms:

Compute (or read from the manifest) the accounts_lt_hash — the lattice hash that summarizes the full account set at the snapshot slot.
Take its BLAKE3 checksum. This is exactly Agave's SnapshotHash::new(checksum) domain — the same value that appears as the archive filename suffix and the same value a known validator advertises over gossip in SnapshotHashes.full.hash.
Assert that computed checksum equals the archive's authoritative checksum (the filename suffix). A wrong or null base fails the guard before the node can vote.

This means the three sources that should agree on the snapshot's identity — the bytes we loaded, the archive's own filename, and what the network gossips — are cross-checked at boot, in a single canonical hash domain (BLAKE3(accounts_lt_hash)). It is the foundation that a future known-validator agreement check (only accept a base that vouched-for validators also advertise) layers on top of.

Snapshot creation (partial — ~30%, not production)¶

Be clear: Vexor cannot yet produce a valid, loadable, cross-client snapshot on its own. This is the least-finished part of the subsystem, and it is honest to call it roughly 30% complete — a scaffold, not a working feature. Do not read the presence of writer code as "snapshot creation works."

What exists¶

An AppendVec writer that serializes the live account store back into Agave's AppendVec record format (accounts.zig writeSnapshotAppendVec / writeSnapshotAppendVecRaw; snapshot.zig writeSnapshotFromAccountsDir).
A saveFullSnapshot scaffold (snapshot.zig) that lays out the directory structure (accounts/, snapshots/<slot>/, a version file) and packages it by shelling out to system zstd (tar --use-compress-program 'zstd -T0' -cf <out>.tar.zst …), mirroring how the extractor untars.
It is designed to be consensus-safe: it reads frozen account state and emits an artifact; it never mutates a bank hash.

Honest note on io_uring. The same io_uring shim referenced in the loading section above is not wired into the write path either — snapshot creation runs on the AppendVec writer and shells out to the system zstd binary, with no io_uring involvement on the read or write side today. Any framing of "io_uring snapshot writes" as a live, cluster-proven capability describes a future optimization target, not current behavior.

What is missing (the reasons it is not production)¶

No in-process zstd encoder. Zig 0.15.2's standard library ships a zstd decoder but no encoder (streaming_decompress.zig notes this explicitly). So creation must shell out to the system zstd binary; there is no native, in-process compression path. Wiring a real encoder (e.g. FFI to system libzstd, or a pure-Zig encoder) is the headline blocker.
Manifest fields are stubbed empty. The bank-manifest serializer (snapshot_manifest.zig) leaves several consensus-relevant fields as documented vecLen(0) stubs — stake_delegations, stake_history, epoch_authorized_voters, and full VoteState bytes. A snapshot with empty stakes is not a valid base another validator could boot from, which is the core reason the produced artifact is not yet a real, loadable snapshot — even though the writer code runs.
Not wired to a live trigger. There is no scheduled "snapshot at slot N" path on the running validator; the writer is a callable scaffold, not an active producer.

A separate status note once listed snapshot creation as a finished "full+incremental .tar.zst writer." That over-states it. Grounded against the actual source (per the project's three-way ground-truth rule), the accurate status is the partial scaffold described here: writer + shell-tar packaging present; encoder and manifest stakes missing; not a valid cross-client snapshot yet.

SnapStream (design + proof-of-concept — a planned differentiator)¶

SnapStream is not a shipped Vexor production feature. It is a real, separately-built project with a shipped distributor, plus a designed cross-client restart-coordination autopilot whose core hard parts are proven in standalone proofs-of-concept. It is documented here because it is one of Vexor's planned differentiators, not because it is live in the validator.

What ships today: the distributor¶

SnapStream as it exists is a fast, verified, cross-client snapshot distributor — a download accelerator. A coordinator publishes a signed manifest (base + incremental snapshots with hashes); any client pulls the chunks in parallel over a custom QUIC transport (with MASQUE / RFC 9298 tunneling for endpoint obfuscation and DDoS resistance). The C-ABI means any validator client — Agave, Frankendancer, Vexor — can integrate it. Core value: snapshots download much faster and are integrity-verified, putting it on the time-to-recovery critical path. This is the entire current scope.

What is designed (not built): the restart autopilot¶

A cluster halt/restart today is a manual, error-prone scramble: every operator must, by hand, agree on the restart slot, build the hard-fork snapshot (with the exact --deactivate-feature-gate account list and cap-change flag), and relaunch with --wait-for-supermajority, --expected-bank-hash, and --expected-shred-version. The network can't resume until ~the supermajority of stake is back, done correctly. The bottleneck is humans.

The proposed autopilot rides SnapStream's existing distribution: a restart coordinator publishes the agreed restart parameters once into a signed control channel, and every opted-in validator — any client, any language — automatically receives, verifies, and applies them: pulls the correct hard-fork snapshot, sets the launch flags, brings up on a fresh ephemeral identity (non-voting), waits for the supermajority/expected-bank gate, and only then hot-swaps to the real staked identity. The hard safety invariant: the real staked key never votes before the gate passes (no premature or equivocating votes). Opt-in only, with autonomy tiers (receive-only / stage-flags / full-autopilot), and fail-closed (any signature/hash/version/bounds check failure halts and alerts).

What is proven (PoC)¶

The two cryptographically hardest pieces are demonstrated in standalone PoCs (results drawn from the SnapStream design notes):

Coordinator authorization via FROST threshold Ed25519. A real 2-of-3 FROST-Ed25519 aggregate signature over a restart manifest verifies under stock ed25519-dalek verify_strict — only a 32-byte group key + 64-byte signature cross the boundary, so any client can verify with an ordinary Ed25519 check; no single key can authorize a restart, and a below-threshold subset cannot forge.
Stake-aware authorization policy. A default policy PoC (role classes + stake-weighted quorum, e.g. "≥5 approvals AND ≥66% of eligible coordinator stake AND faction diversity") passes its test suite, including anti-capture cases (a whole admin faction alone is blocked) and live-tunable policy.

Honest positioning¶

Agave's wen_restart (SIMD-0046) already automates agreeing which slot to restart from, but only within one client. SnapStream's wedge is the unsolved part: a client-agnostic control plane plus the fast distribution channel on the time-to-supermajority critical path. It complements wen_restart; it does not replace consensus. It is a network-resilience public good — valuable precisely because it is cross-client — and a planned, not shipped, capability.

Configuration¶

Snapshot loading behavior — the part that's live today — is controlled by environment flags read at boot. For the full operational walkthrough (what happens at each boot step, and when to choose fresh vs. reuse), see the deploy runbook's snapshot policy.

Flag	Effect	Default
`VEX_FRESH_SNAPSHOT`	`1` re-extracts the snapshot every boot and, if the local tarball is more than `VEX_SNAP_MAX_AGE_SLOTS` behind the live tip, downloads a fresh near-tip base+incremental pair. Working account cache, ledger, and tower state are wiped either way — this is the production-correct path, since it never inherits possibly-corrupt local state.	recommended `1`
`VEX_SNAP_MAX_AGE_SLOTS`	Freshness threshold for the check above, in slots. `0` forces a refetch on every boot; a very large value effectively preserves whatever local snapshot is already there.	`25000` (~2.8 hours)
`VEX_REUSE_SNAPSHOT`	`1` boots from the existing local `extracted-*` snapshot without deleting or re-downloading it — the safe way to reproduce a specific slot for carrier diagnosis or regression repro. Working account cache, ledger, and tower are still wiped and rebuilt from that snapshot.	`0` (off)

A minimal deploy invocation carrying the fresh-snapshot flag looks like this (everything else self-defaults):

VEX_BINARY=<pinned> VEX_ENABLE_AFXDP=1 VEX_FRESH_SNAPSHOT=1 VEX_PARALLEL_EXEC=1 VEX_PARALLEL_EXEC_CORES=25,26 VEX_TASKSET_CORES=<mask> bash deploy.sh dev

Full definitions, plus the rest of the deploy-time environment surface, live in the environment variable reference. Snapshot creation has no operator-facing flags yet — see Snapshot creation above for why.

Status (2026-07-11)¶

Loading — DONE, live, proven. mmap + parallel AppendVec parser boots ~86.7M accounts as the validator's account state; correctness is proven by the node voting bank-exact. (io_uring is not wired — the live path is mmap + thread-per-core parsing.)
Verification — DONE, live. Full-snapshot lt-hash guard runs unconditionally at boot: BLAKE3(accounts_lt_hash) must equal the archive's authoritative checksum (Agave SnapshotHash::new domain).
Creation — PARTIAL (~30%), not production. AppendVec writer + shell-tar packaging exist; missing an in-process zstd encoder (Zig 0.15.2 has no zstd encoder), several stubbed-empty manifest fields (stake_delegations, stake_history, epoch_authorized_voters, full VoteState), and a live trigger. The produced artifact is not yet a valid loadable cross-client snapshot.
SnapStream — DESIGN + PoC, not shipped. Distributor ships (cross-client, signed-manifest, QUIC/MASQUE); the restart autopilot is designed, with the FROST-Ed25519 threshold-sign and stake-aware authorization PoCs proven. A planned differentiator, not a live production feature.

Roadmap¶

Wire a real zstd encoder and fill the stubbed manifest stake/vote fields → first valid Vexor-produced snapshot; then a scheduled creation trigger.
Known-validator agreement check layered on the existing lt-hash guard (only accept a base that vouched-for validators also advertise).
io_uring batch reads as a loader optimization (current loader is already proven without it).
Build out the SnapStream restart autopilot from PoC to an audited, opt-in, cross-client tool.

Source references: src/vex_store/parallel_snapshot.zig (mmap + parallel loader), src/vex_store/accounts.zig and src/vex_store/snapshot.zig (AppendVec writer, saveFullSnapshot), src/vex_store/snapshot_manifest.zig (manifest, stubbed fields), src/vex_svm/bootstrap.zig (verifyBaseLtHashAgainstArchive lt-hash guard).