Skip to content

Tuning

Docs-site note (rewritten 2026-07-11): Part of the Vexor documentation site (Anza-style). This page covers the performance knobs that are actually worth an operator's attention — what each does, its default, and the telemetry line to watch when you change it. The exhaustive, source-grounded list of every environment variable (including the ones the binary now sets for you) lives on the Environment Variable Reference; this page is the curated "what to tune and what to watch" companion to it, not a duplicate of it.

The short version: most of this is baked in now

As of the current binary, Vexor bakes its own proven performance defaults at startup — parallel execution, the FEC-dedup cache, the ledger write path, and about twenty-three other VEX_* variables are set to their proven values automatically, unless you've already set them yourself. See the baked necessity set for the full table.

That leaves a small, genuinely host-specific surface: which binary to run, whether this NIC can do AF_XDP, which cores the parallel-execution workers live on, and the process-wide core mask. The reference deploy looks like this:

VEX_BINARY=<pinned> \
  VEX_ENABLE_AFXDP=1 \
  VEX_FRESH_SNAPSHOT=1 \
  VEX_PARALLEL_EXEC=1 VEX_PARALLEL_EXEC_CORES=25,26 \
  VEX_TASKSET_CORES=<mask> \
  bash deploy.sh dev

(In practice this is appended with --identity, --vote-account, and --public-ip — see the minimal recipe for a fully worked example.)

VEX_PARALLEL_EXEC=1 is written out explicitly above even though it's also baked on by default — it's harmless either way, and spelling it out on the command line makes the intent visible in process listings and unit files. Everything below this section is either one of those genuinely host-specific knobs, or a baked-on default that's still worth understanding and knowing how to override for a specific diagnostic session.

The guiding rule hasn't changed: a perf knob is either proven bank-hash-neutral (safe to leave on) or it stays off. None of the knobs on this page should ever change the bank hash; if one ever correlates with divergence, the correct move is to turn it back off (=0) and report it, not to tune around it.

Transaction execution

VEX_PARALLEL_EXEC

Arms the wave-barrier parallel transaction execution path (see Enhancements). Requires a binary built with -Dparallel_exec (included in -Dprod).
Effect =1 (or unset — this is baked on) arms the WavePool; conflict-free native transactions execute in parallel on the worker cores, then commit serially in wave order. Explicit 0 runs the serial path, byte-identically.
Default Baked on (1) as of the current binary. Proven bank-exact over ~1,600 offline-replay slots plus continued live operation.
Risk Bank-hash-neutral by construction (commutative freeze fold + DAG write→read exclusion). Throughput-only.

VEX_PARALLEL_EXEC_CORES
Effect Comma-separated list of cores for the persistent wave workers, e.g. 25,26.
Default Not baked — hardware-specific, so you set it explicitly. A pair of cores on a cold CCX, never on CCX0 (cores 0–3 are OS/kernel reserved) and never colliding with a hot tile (verify 8–15, replay 16, produce 20, gossip 24). 25,26 is the proven choice on the reference 32-core box — see Hardware.
What to measure During catch-up, watch that the node stays CURRENT; if parallel catch-up lags, confirm the wave cores are actually free (the deploy auto-carves them out of the floater mask when VEX_PARALLEL_EXEC is armed).

Snapshot warm-up

VEX_FRESH_SNAPSHOT
Effect =1 re-extracts the snapshot on every boot and, if the local one is more than VEX_SNAP_MAX_AGE_SLOTS behind the live tip, downloads a fresh near-tip base + incremental pair before launch.
Default recommendation On (1). This is the production-correct boot path — it never inherits possibly-corrupt local state, and the loader is parallelized (mmap + worker-thread pool), so a near-tip incremental boot is tens of seconds, not a slow single-threaded unpack. See the snapshot policy for the full contract, including the reuse-mode alternative for pinning a specific slot.
What to measure Boot-to-CURRENT time in the deploy log; if it's unexpectedly long, check whether a fresh download was actually needed (VEX_SNAP_MAX_AGE_SLOTS, default 25000 slots ≈ 2.8 hours) or the entrypoint fetch itself is slow.

VexLedger (blockstore)

These knobs control VexLedger. The storage path is always active when the ledger is enabled (VEX_LEDGER=1, baked on by default); these tune durability vs throughput and disk footprint. Full table on the VexLedger Configuration section.

VEX_LEDGER_FSYNC_EVERY
Effect fsync the ledger once per N completed slots. 1 = fsync every slot (the durable default); higher N = fewer fsyncs = more write throughput, at the cost of losing the last up-to-N slots of shreds on an unclean shutdown.
Default 1 (every slot). Shreds are best-effort and re-fetchable from the network, so a higher value is a reasonable throughput trade if persist latency is a concern — but 1 is the durable baseline.
What to measure persist_us(avg/max) in the [LEDGER-TILE-STATS] line (below). The fsync dominates per-slot persist latency, so this is the knob that moves that number.

VEX_LEDGER_MAX_BYTES
Effect Bound the on-disk ledger to N bytes (--limit-ledger-size semantics). The oldest whole sealed segment is evicted with an O(1) unlink() — no compaction, instant space reclaim. Takes precedence over VEX_LEDGER_KEEP_SLOTS if both are set.
Default Baked on at 107374182400 (100 GiB) as of the current binary — no longer unbounded by default. Set it explicitly to the size budget of your ledger drive minus headroom if 100 GiB isn't the right number for your box.
What to measure The pruned=<seg>/<MiB> field in [LEDGER-TILE-STATS], and free space on the ledger drive — confirm eviction is keeping pace with the disk budget.

Networking

VEX_ENABLE_AFXDP
Effect =1 enables the AF_XDP zero-copy ingest datapath (with --enable-af-xdp --xdp-zero-copy). Off uses the conventional socket RX path.
Default Not baked — hardware-specific, so you set it explicitly per host. But AF_XDP must run on a NIC queue that does not also carry management/SSH traffic — enabling XDP on the live-cabled port link-resets it. Treat first-time enablement as a maintenance-window operation with console access. See Network setup.
What to measure The [AFXDP-PHASE] line (loops/s, recv/zc-recv latency, ring_depth_max, zc_overrun): zc_overrun≈0 and ring_depth_max≈0 at steady state means RX is keeping up.

VEXOR_ED25519_FEC_DEDUP
Effect =1 arms the FEC-set ed25519 dedup cache (skip re-verifying a signature already verified for the same FEC set). Requires a binary built with -Dfec_dedup (included in -Dprod).
Default Baked on (1) as of the current binary. Bank-hash-neutral — it only ever skips a verify that would have succeeded, so it cannot change what is admitted.
What to measure Steady-state catch-up health (the node should stay CURRENT and bank-exact); a healthy dedup path shows zero false rejects.

Core placement

VEX_TASKSET_CORES
Effect The process-wide taskset core mask. Defines which cores the validator may run on; the per-tile pins (from src/vex_topo.zig) place specific threads within that mask, and the deploy also applies a cgroup-v2 cpuset wall-off.
Default Not baked — hardware-specific, so you set it explicitly. Reserve CCX0 (cores 0–3) for the OS/kernel and place the validator on the remaining cores. With AF_XDP the proven mask on the reference box is 1-3,5-27; if parallel-exec catch-up lags, carve the wave cores out explicitly (1-3,5-24,27) so floaters never preempt cores 25,26. The deploy auto-carves the wave cores when VEX_PARALLEL_EXEC=1.
What to measure The deploy-time /proc tiling verifier — it prints the live thread→core map and flags any consensus-core floater. A clean verifier output is the gate; an unexpected floater on a hot core is the signal to fix the mask.

Telemetry — the data source

Vexor emits structured per-window stats lines that are the ground truth for tuning. Read these from the live log rather than guessing.

[LEDGER-TILE-STATS]

The primary signal for the ledger persist path. One line per stats window:

[LEDGER-TILE-STATS] slots=… enq=… applied=… dropped=… ring_high_water=…/<cap> persist_us(avg/max)=…/… pruned=…seg/…MiB
Field What it tells you
persist_us(avg/max) Per-slot persist latency (serialize meta + append + fsync). The fsync dominates — this is the VEX_LEDGER_FSYNC_EVERY tuning signal. ~2 ms is the healthy range on the reference box.
ring_high_water / cap Peak occupancy of the producer→consumer ring since the last window, against its capacity. If ring_high_water approaches cap, the persist consumer is falling behind the completion threads (disk backpressure) — raise VEX_LEDGER_FSYNC_EVERY or check the ledger drive.
dropped Ring-full / backpressure drops. The ledger is best-effort and off the consensus path, so a drop self-identifies a missing shred on replay but never affects the bank hash. A non-zero dropped means the persist tile is overloaded — investigate the ledger drive and fsync cadence. It should be 0 at steady state.
slots / enq / applied Completed-slot count and enqueued-vs-applied message counts — sanity that the tile is draining everything it is handed.
pruned=<seg>/<MiB> Cumulative segments/space reclaimed by VEX_LEDGER_MAX_BYTES pruning.

Other useful lines

  • [AFXDP-PHASE] — RX datapath health (see VEX_ENABLE_AFXDP above): zc_overrun and ring_depth_max should be ≈0 at steady state.
  • [QUIC-VOTE-STATS] — vote submission path (QUIC vs UDP counts, pool size/evictions, cache hit/miss), logged roughly every 10 s.

A safe tuning workflow

  1. Start from the reference deploy above — the baked defaults plus that handful of host-specific variables are the proven starting point.
  2. Change one knob at a time, then watch the telemetry for a full stats window before judging.
  3. Gate on correctness, not just speed. The node must stay CURRENT and bank-exact after any change. If a knob ever correlates with divergence, turn it off — it should not, and that is a bug to report, not to tune around.
  4. For durability vs throughput (VEX_LEDGER_FSYNC_EVERY), trade only as far as persist_us and ring_high_water justify, and remember shreds are re-fetchable — so the only loss from a higher fsync cadence is a brief re-fetch on unclean restart.
  5. To disarm a baked default for a diagnostic session, set it explicitly to 0 on the command line — an explicit value always wins over the bake, no rebuild required. See Environment Variable Reference for the full baked list.

See also