Architecture Overview¶

Docs-site note (rewritten 2026-07-11): Part of the Vexor documentation site (Anza-style). This page gives the whole-system picture: how the validator boots, how a packet becomes a frozen, voted-on block, and how the major subsystems fit together. Each subsystem links to its own page. This rewrite re-anchors the client-fidelity claim to the current ground truth (Agave v4.2.0-beta.0, the newest testnet release) and to the vote program's current status (Vexor-authored, now the live executor) — earlier drafts of this page referenced an older rc.1 anchor and a build-flag story that has since changed. Claims are grounded in the live source tree and the testnet deployment; where something is partial or gated, it says so.

What Vexor is¶

Vexor (binary vex-fd) is an independent Solana validator client written from scratch in Zig (Zig 0.15.2), operated by QubeStake (identity 3J2jADiEoKMaooCQbkyr9aLnjAb5ApDWfVvKgzyK2fbP, with ~235k SOL of privately delegated testnet stake). It speaks the Solana protocol, executes the Solana runtime, and votes in consensus alongside Agave and Firedancer nodes. It runs on testnet today: voting CURRENT and bank-exact (its per-slot bank_hash matches the cluster's), with AF_XDP zero-copy ingest, native QUIC votes, and its own Zig-native blockstore.

Two honest limitations, stated up front rather than buried: Vexor is testnet-only — mainnet readiness is not claimed anywhere in this documentation. And it is versioned 0.9.x, pre-production — its gossip client id (86) is not yet registered upstream, so public explorers show it as Unknown(86). Version 1.0.0 is reserved for the production-grade milestone.

Vexor is a third independent client, not a fork. The runtime, the sBPF virtual machine, the accounts database, the consensus/replay engine, the network stack, the blockstore, and — as of the pure-Zig crypto rewrite — the leaf-level cryptography are all clean-room Zig — validated byte-for-byte against the Agave reference implementation, currently anchored to the newest testnet release, v4.2.0-beta.0 (the anchor moves forward as new testnet releases ship), with Firedancer and Sig used as engineering references. The canonical production build (-Dpure_zig) links zero Firedancer code at runtime: ed25519, BLAKE3, BN254/alt_bn128, and Poseidon all run through Vexor's own src/vex_crypto/, verified byte-exact against known-answer test vectors and live cluster parity. Firedancer's Ballet AVX-512 crypto libraries have been removed from the tree outright (2026-07-12) — there is no build configuration that links them anymore.

The vote program — the validator's single most-executed code path — is Vexor-authored: a from-scratch rewrite (July 2026), derived from Agave 4.2 semantics, and it is the sole, unconditional live executor on testnet today — no build flag selects it, and no alternate path exists to select instead. A Sig-derived implementation served as the differential-test oracle during the rewrite and has been fully removed from the tree (Stage 8, 2026-07-12, −31,181 lines) now that the rewrite has proven out against it. See The Vote Program for the full story and the measured numbers.

The pinned-tile model¶

Vexor runs as a set of tiles — dedicated threads, each pinned to a specific CPU core — in the Firedancer style. This is the single most important structural decision in the architecture, and it shapes everything else.

A validator's hot paths are latency-bound, not throughput-bound in the usual sense: replay must execute a block and hash its Proof-of-History faster than the cluster produces the next one, or the node falls behind. If those threads float across cores at the scheduler's whim, they suffer cache misses and preemption stalls exactly when they can least afford them. Pinning each role to its own core means:

No preemption of a consensus-critical thread by background work.
Cache locality — a tile keeps its working set warm in its core's L1/L2 and its CCX's shared L3, instead of having it evicted by an unrelated thread.
CCX-aware isolation — on the reference EPYC (8 Core-Complexes, 32 MiB L3 each), cold/background work (the ledger tile, build relief) is fenced onto a separate CCX so it can never evict the hot working set.
Determinism — the same work runs on the same core every time, which makes performance reproducible and divergence forensics tractable.

The tile→core assignment is a single declarative table (src/vex_topo.zig), with comptime invariants that turn an accidental core-number edit into a build error rather than a silent mis-pin.

Core-pinning layout¶

The live testnet layout on the 32-core EPYC 9374F:

Tile	Core(s)	Role
`recv`	5	TVU receive / network front-end (shred ingest)
`quic`	6	QUIC TPU pump (transaction + vote ingest)
`verify`	8–15	Signature-verify worker pool (8 workers)
`replay`	16	Replay — the consensus heart (execute + freeze + vote)
`produce`	20	Block production (pack + PoH), leader-gated
`gossip`	24	Gossip / CRDS
`rpc`	27	RPC HTTP listen loop (diagnostic, bursty — off the hot pipeline)
`txsend`	28	Vote sender
`sysvar`	29	Sysvar refresh
`repair`	30	Repair / control
wave workers	25, 26	Parallel transaction execution (WavePool)
ledger tile	4	VexLedger persist consumer (cold core, off the consensus path)
OS / relief	0, 1–3	OS + main thread (0); dynamic-relief / build pool (CCX0 1–3)

Principles: consensus-critical tiles (verify, replay, produce, txsend) sit on cache-warm cores and never share a core with background work; the VexLedger consumer runs on a cold core, entirely off the consensus path, so persistence can never stall replay or voting; builds are fenced onto CCX0 (and, by deploy discipline, the node is stopped before a release build anyway). See Hardware for how this maps onto the silicon and Network setup for the pinning verifier and the cgroup cpuset.

Data flow — packet to vote¶

A shred enters at the NIC and ends as a vote (and, when Vexor is leader, as a broadcast block). The path crosses tile boundaries via lock-free rings, so each stage runs at its own pace on its own core.

                     NIC (Mellanox ConnectX-6 Dx)
                              │
        ┌─────────────────────┴─────────────────────┐
        │ shreds: UDP / AF_XDP zero-copy             │ transactions + votes: QUIC
        ▼  (recv tile, core 5)                       ▼  (quic tile, core 6)
   ┌──────────────┐                            ┌──────────────┐
   │  packet      │                            │  QUIC streams │
   │  ingest      │                            │  → tx decode  │
   └──────┬───────┘                            └──────┬───────┘
          ▼                                           │
   ┌──────────────┐   sigverify (verify pool,         │
   │  shred        │   cores 8–15) + ed25519 FEC-set   │
   │  sigverify    │   dedup cache                     │
   └──────┬───────┘                                    │
          ▼                                            │
   ┌──────────────┐   Reed-Solomon recovery of any     │
   │  FEC / shred  │   missing shreds; merkle-root      │
   │  assembly     │   verify; repair fills holes       │
   └──────┬───────┘                                    │
          ▼                                            ▼
   ┌─────────────────────────────────────────────────────────┐
   │  REPLAY (core 16) + WavePool parallel exec (25, 26)       │
   │   • parse block, execute every transaction in the sBPF VM │
   │   • apply account mutations to AccountsDb                  │
   │   • update sysvars, PoH                                    │
   └──────────────────────────┬──────────────────────────────┘
                              ▼
                   ┌────────────────────┐
                   │  BANK FREEZE         │  bank_hash = f(parent_hash,
                   │  bank_hash + lt_hash │  signature_count, PoH, accounts lt_hash)
                   └─────────┬───────────┘
                             ▼
              ┌──────────────┴───────────────┐
              ▼                               ▼
       ┌─────────────┐                 ┌──────────────────┐
       │  VOTE        │                 │  (LEADER ONLY)    │
       │  (txsend 28) │                 │  block production │
       │  via QUIC/   │                 │  produce (20) +   │
       │  UDP relay   │                 │  Turbine broadcast│
       └─────────────┘                 └──────────────────┘
                             │
                             ▼
                   ┌────────────────────┐
                   │  VexLedger persist   │  (ledger tile, cold core 4)
                   │  shreds + metadata   │
                   └────────────────────┘

Step by step:

Ingest. Shreds arrive as UDP datagrams, delivered zero-copy through AF_XDP on the Mellanox NIC (with a kernel-UDP fallback). Transactions and TPU votes arrive over QUIC. (Shreds are deliberately not QUIC — they are tiny, erasure-coded packets where QUIC's per-stream overhead would only hurt.)
Sigverify. The verify pool checks shred signatures. An ed25519 FEC-set dedup cache verifies the first shred of each FEC set and lets its ~63 identical-signature siblings skip the expensive ed25519 step.
FEC / assembly. Missing shreds are recovered with Reed-Solomon erasure coding; merkle roots are verified; repair requests fill any remaining holes from peers.
Replay / execution. Each completed slot's block is replayed: every transaction executes through Vexor's runtime — native programs, including the Vexor-authored vote program, plus the sBPF interpreter for deployed programs — and account mutations land in the accounts database. Independent transactions can execute in parallel on the WavePool workers.
Bank freeze. When a slot is fully replayed, the bank is frozen and its bank_hash computed — a function of the parent hash, the signature count, the PoH hash, and the accounts lt-hash (a homomorphic lattice-hash over the account set). A matching bank_hash is what "bank-exact" means.
Vote. Vexor signs a vote for the frozen slot and submits it over QUIC (with a UDP relay fallback).
Block production (leader only). During its leader window, Vexor packs a block, extends PoH, shreds it (Reed-Solomon encoder), and broadcasts via Turbine down the stake-weighted tree. Today this produces empty blocks only — transaction-bearing production is gated (see Subsystem map below).
Persistence. Shreds and per-slot metadata are streamed to VexLedger on a cold core, off the consensus path.

Subsystem map¶

Subsystem	What it does	Page
Networking	AF_XDP zero-copy ingest, native QUIC, Turbine, Repair, Gossip/CRDS, FEC	Networking
Runtime / SVM	Executes transactions: native programs + the sBPF interpreter, syscalls, the full feature-gated runtime	Runtime & SVM
Vote Program	The Vexor-authored, from-scratch vote-instruction executor — the validator's most-executed code path	The Vote Program
Accounts DB	The account store: in-memory index over mmap'd AppendVec storage, the lt-hash	(accounts page — planned)
Consensus / Replay	Replay loop, fork choice, tower/lockouts, vote state, bank freeze	Consensus
VexLedger	The Zig-native append-segment blockstore (drop-in for Agave's RocksDB)	VexLedger
Snapshots	Loads real Agave snapshots bank-exact (mmap); creation is partial; SnapStream is a design	Snapshots

Notes on subsystem status (honest framing)¶

sBPF VM. Vexor has its own clean-room sBPF virtual machine (interpreter.zig, a mirror of solana-sbpf), with verifier, ELF loader, and memory regions. It executes bank-exact. It is an interpreter, which is at Firedancer parity — Firedancer also interprets in its consensus path; the JIT is an Agave-only feature, not a gap.
Vote program. The rewrite executes vote instructions in ~1.9–2.0 µs, versus ~8.9 µs for the retired Sig-derived transplant it replaces — a 4.7× speedup, measured over 990,000+ live instructions with byte-identical results between the two implementations. See The Vote Program.
Accounts DB. The account set is memory-mapped from the snapshot rather than copied to the heap, so most of RAM serves as page cache backing those mmaps.
Snapshots. Vexor loads real Agave snapshots bank-exact. Snapshot creation is partial (the AppendVec writer exists; the manifest serializer, tar packaging, and a zstd encoder are not yet wired) — Vexor is not yet a self-sufficient snapshot producer.
Block production. Leader block production is proven (blocks accepted by the cluster), today producing empty blocks at roughly 97%+ of leader slots produced and cluster-accepted; transaction-bearing block production (cost model + account-lock scheduling + sigverify pre-include + flipping the inclusion gate) is gated and under active development — it is not yet a shipped, documented-as-working feature.

Build and deploy shape¶

Vexor compiles to a single static binary. A production build is one command:

zig build -Dprod -Dpure_zig -Dcpu=znver4 --release=safe

-Dprod bundles the 12 build flags a production voting deploy needs (vote-program execution, leader mode, parallel execution, and the rest) into one switch — each stays individually overridable, and the build is golden-gated as behaviorally identical to assembling the same 12 flags by hand. -Dpure_zig selects the pure-Zig crypto backend (Vexor's own ed25519/BLAKE3/BN254/Poseidon, in place of -Dprod's default Firedancer Ballet crypto flags) and is what the canonical production binary ships with. --release=safe is mandatory: the deploy pipeline rejects an oversized binary (a debug build fails the deploy-time >50MB check) rather than risk launching one. Substitute your own -Dcpu target for the reference box's znver4 (Zen 4 EPYC). See Building Vexor for the full flag table.

At deploy time, the pipeline runs hard, binary-inspecting consensus guards — not "did the build succeed" but "does this artifact actually contain the paths a validator must not run without" — and refuses to launch a binary missing the compiled-in vote-execution or BN254 leaf-crypto markers. The deploy cadence is strictly stop → build → deploy: the node is brought down before any release build. See Deploying Vexor for the full ordered contract, and the Build-Flag Reference / Environment Variable Reference for the complete flag and env surface — most of what a deploy needs today is baked into the binary by default, leaving a small, honest operator surface (which binary, which cores, whether AF_XDP is available).

Status (2026-07-11)¶

Live on testnet: voting CURRENT, bank-exact, at roughly 98.7% of maximum vote credits (measured against the theoretical 16-credits/slot ceiling, on par with the Agave reference node run alongside it). AF_XDP zero-copy ingest, native QUIC vote submission (with a UDP relay fallback), FEC-set dedup, and VexLedger as the blockstore.
Block production: roughly 97%+ of leader slots produced and cluster-accepted, empty blocks only; transaction-bearing production is gated and not yet live.
Vote program: the Vexor-authored rewrite is the sole, unconditional live executor; the Sig-derived component that served as its differential-test oracle has been fully removed from the tree (Stage 8, 2026-07-12).
Pinned-tile model: the full layout above is the deployed topology, enforced by comptime invariants.
Parallel execution: WavePool (cores 25, 26) is in the production build and proven bank-exact vs serial.
Metrics: Vexor submits to the Solana metrics database (Agave-compatible schema) plus Vexor-specific vexor-* measurements — see Observability.
Open work: transaction-bearing block production, snapshot creation, a VEX_MODE (full / vote / rpc) runtime switch (spec'd, not yet implemented), and a set of latent feature-gate ports (dormant until the cluster activates them) — none is an active consensus divergence.