VMess: keys and authentication

VMess in Etemenanki is the AEAD-only variant: every connection starts with a 16-byte authentication id, followed by a request header sealed with AES-128-GCM. This page covers the cryptographic half of that protocol: how each key is derived from the user’s UUID, how the authentication id is built and checked, how the header envelope is sealed and opened, the in-crate SHAKE128, the typed key wrappers, and AccountValidator, which matches an incoming id against every configured user and rejects replays.

The byte layout of the inner request header, the chunk framing of the body and the server state machine are on VMess: wire format and core. Read this page before you change a key derivation, a salt, the time window or the replay set. The derivations and salts must stay byte-compatible with Xray. Every unit test seals and opens with this crate’s own code, so only the Xray interoperability tests notice a wrong salt or KDF step, and those tests are skipped when go is not installed.

Responsibilities

File	Symbol(s)	Responsibility
protocols/src/vmess/aead.rs	kdf, kdf16, cmd_key	The recursive HMAC-SHA256 KDF and the account’s cmd key.
protocols/src/vmess/aead.rs	GcmKey, GcmNonce, ConnNonce	Role-named constructors for every header key and nonce.
protocols/src/vmess/aead.rs	create_auth_id, auth_id_decipher, decode_auth_id_dec	Sealing and opening the 16-byte authentication id.
protocols/src/vmess/aead.rs	seal_vmess_aead_header, open_vmess_aead_header, open_vmess_aead_header_slice	The sealed request-header envelope.
protocols/src/vmess/aead.rs	Shake128	A self-contained SHAKE128 XOF for chunk-length masking and padding.
protocols/src/vmess/keys.rs	CmdKey, BodyKey, BodyIv, AuthId, AuthIdPlain	Typed 16-byte key material, so the compiler rejects swapped keys.
protocols/src/vmess/accounts.rs	AccountValidator, Account, MatchedAccount	User matching, the time window, replay protection and scan-order tuning.

The module does not parse the inner header, frame the body or drive the connection. protocols/src/vmess/protocol.rs and protocols/src/vmess/framing.rs call into it for that, and protocols/src/vmess/core.rs → VMessCore calls AccountValidator::authenticate and open_vmess_aead_header_slice from its handshake states.

Key derivation

The cmd key

Every per-user secret starts from the account’s cmd key, the MD5 of the 16 raw UUID bytes followed by a fixed magic string:

const CMD_KEY_MAGIC: &[u8] = b"c48619fe-8f02-49e0-b9e9-edf763e17e21";

pub fn cmd_key(uuid: &uuid::Uuid) -> CmdKey

CmdKey = MD5(uuid.as_bytes() || CMD_KEY_MAGIC). No other derivation reads the UUID; the validator keeps it only to report which account matched.

The KDF chain

kdf is VMess’s recursively nested HMAC. The salt SALT_VMESS_AEAD_KDF ("VMess AEAD KDF") keys the innermost HMAC-SHA256, and each element of path adds one more HMAC level whose hash function is the level below it:

pub fn kdf(key: &[u8], path: &[&[u8]]) -> [u8; 32]
pub fn kdf16(key: &[u8], path: &[&[u8]]) -> [u8; 16]
fn hmac_chain(keys: &[&[u8]], level: usize, msg: &[u8]) -> [u8; 32]

For a path p1 … pn:

Level	Keyed with	Hash function inside the HMAC
H0	"VMess AEAD KDF"	SHA-256
H1	p1	H0
Hi	pi	H(i-1)
Hn	pn	H(n-1)

kdf(key, path) returns Hn(key), the full 32 bytes. kdf16 truncates it to the first 16. hmac_chain treats every level as a hash with a 64-byte block and a 32-byte output: a key longer than 64 bytes is first hashed with the level below (SHA-256 at level 0). None of the salts, auth ids or nonces fed to it exceed 64 bytes, so that branch exists for correctness only.

flowchart LR
  K["input key (cmd key or body key/IV)"] --> Hn
  subgraph chain["kdf(key, p1..pn)"]
    Hn["Hn: HMAC keyed pn"] -->|"inner and outer hash"| H1["H1: HMAC keyed p1"]
    H1 -->|"inner and outer hash"| H0["H0: HMAC-SHA256 keyed 'VMess AEAD KDF'"]
  end
  Hn --> Out["32-byte digest"]
  Out --> K16["kdf16: first 16 bytes (keys)"]
  Out --> N12["first 12 bytes (GCM nonces)"]

Each level calls the level below twice (inner and outer hash), so a path of length n costs 2^(n+1) SHA-256 invocations. The auth-id key (n = 1) costs 4. Each request-header key or nonce (n = 3) costs 16, so opening one header derives four values for 64 SHA-256 calls. This is why AccountValidator derives the auth-id ciphers once per user at build time rather than per connection.

Salts

All salts are private &[u8] constants in protocols/src/vmess/aead.rs, copied from Xray’s proxy/vmess/aead/consts.go:

Constant	Value	Derives
SALT_VMESS_AEAD_KDF	VMess AEAD KDF	The innermost HMAC key of every kdf call.
SALT_AUTHID_ENC_KEY	AES Auth ID Encryption	The AES-128 key for auth ids, from the cmd key.
SALT_HEADER_PAYLOAD_LEN_AEAD_KEY	VMess Header AEAD Key_Length	GcmKey::request_len
SALT_HEADER_PAYLOAD_LEN_AEAD_IV	VMess Header AEAD Nonce_Length	GcmNonce::request_len
SALT_HEADER_PAYLOAD_AEAD_KEY	VMess Header AEAD Key	GcmKey::request_payload
SALT_HEADER_PAYLOAD_AEAD_IV	VMess Header AEAD Nonce	GcmNonce::request_payload
SALT_AEAD_RESP_HEADER_LEN_KEY	AEAD Resp Header Len Key	GcmKey::response_len
SALT_AEAD_RESP_HEADER_LEN_IV	AEAD Resp Header Len IV	GcmNonce::response_len
SALT_AEAD_RESP_HEADER_PAYLOAD_KEY	AEAD Resp Header Key	GcmKey::response_payload
SALT_AEAD_RESP_HEADER_PAYLOAD_IV	AEAD Resp Header IV	GcmNonce::response_payload

Where every key comes from

flowchart TB
  uuid["user UUID"] -->|"MD5 with CMD_KEY_MAGIC"| cmd["CmdKey"]
  cmd -->|"kdf16: AES Auth ID Encryption"| aid["auth-id AES-128 key"]
  cmd --> req["request header: GcmKey and GcmNonce request_len, request_payload"]
  authid["AuthId (wire)"] --> req
  conn["ConnNonce (wire)"] --> req
  bk["BodyKey (random, inside header)"] -->|"SHA-256, first 16"| rbk["response BodyKey"]
  biv["BodyIv (random, inside header)"] -->|"SHA-256, first 16"| rbiv["response BodyIv"]
  rbk --> resp["response header: GcmKey response_len, response_payload"]
  rbiv --> respn["response header: GcmNonce response_len, response_payload"]
  biv --> shake["request-direction Shake128"]
  rbiv --> rshake["response-direction Shake128"]

The request-header keys depend on the cmd key and on two values that travel in the clear: the auth id and the connection nonce. The body key and IV are chosen at random by the client (BodyKey::random, BodyIv::random in protocols/src/vmess/session.rs → OutboundSession::new) and carried inside the sealed header. Both ends derive the response direction from them with protocols/src/vmess/protocol.rs → RequestSession::response, which calls BodyKey::response and BodyIv::response: the client when it builds the session, the server after it has opened the request header. The response-header keys and nonces are derived from those response values. How the body key becomes an AES-128-GCM or ChaCha20-Poly1305 chunk cipher is covered on VMess: wire format and core.

The authentication id

The auth id is the first 16 bytes of every VMess connection: one AES-128-ECB block, encrypted with the per-user key kdf16(cmd_key, [SALT_AUTHID_ENC_KEY]).

Plaintext layout

Offset	Size	Field	Meaning
0	8	time	Seconds since the Unix epoch, signed, big-endian (i64::to_be_bytes).
8	4	random	Four random bytes, so two ids in the same second differ.
12	4	CRC32	crc32fast::hash (IEEE CRC-32) of bytes 0 to 11, big-endian.

Signatures

pub fn now_unix() -> i64

pub fn auth_id_cipher(cmd_key: &CmdKey) -> Aes128
pub fn create_auth_id_with_cipher(cipher: &Aes128, time: i64) -> AuthId
pub fn create_auth_id(cmd_key: &CmdKey, time: i64) -> AuthId

pub fn auth_id_decipher(cmd_key: &CmdKey) -> Aes128Dec
pub fn decode_auth_id_dec(cipher: &Aes128Dec, authid: &AuthId) -> AuthIdPlain

create_auth_id_with_cipher fills the block, computes the CRC over the first 12 bytes and encrypts it in place. The client calls create_auth_id(cmd_key, now_unix()) through seal_vmess_aead_header. now_unix returns 0 if the system clock reads earlier than the epoch.

The server side uses the decrypt-only Aes128Dec: it holds only the decryption round keys, half the size of a full Aes128, which matters when one is resident per user. decode_auth_id_dec decrypts one block and returns an AuthIdPlain.

How the server checks an id

protocols/src/vmess/accounts.rs → decipher_matches is the per-user test run by the scan:

1. Decrypt the block with the user’s Aes128Dec.
2. AuthIdPlain::crc_ok(): the stored CRC32 must equal the CRC32 of the first 12 bytes. Decrypting with the wrong user’s key produces noise, which passes this check with probability 2^-32. Passing it is what identifies the owning account.
3. AuthIdPlain::timestamp() must not be negative.
4. now.checked_sub(t) must not overflow, and its absolute value must be at most AUTHID_WINDOW_SECS (120). The bound is inclusive and symmetric, so a client clock up to 120 s fast or slow is accepted.

now is the server’s wall clock in seconds. VMessCore::new takes it as a now: fn() -> i64 parameter, and the core calls (self.now)() when the first 16 bytes arrive. The app (app/src/serve.rs), katana (src/serve.rs) and every current test pass aead::now_unix.

The header AEAD

Sealed request header

seal_vmess_aead_header wraps the inner request header (built by encode_request_header in protocols/src/vmess/protocol.rs) in this envelope:

Field	Size	Meaning
auth id	16	create_auth_id(cmd_key, now_unix()), sent in the clear.
sealed length	18	The inner header length as a big-endian u16, sealed with AES-128-GCM: 2 bytes of ciphertext plus the 16-byte tag.
connection nonce	8	ConnNonce::random(), sent in the clear.
sealed payload	L + 16	The inner header (L bytes), sealed with AES-128-GCM, plus the tag.

Both seals use the 16 auth-id bytes as associated data, so the length and the payload are bound to the auth id they arrived with.

Keys and nonces

Every key is kdf16 and every nonce is the first 12 bytes of a full kdf digest (GcmNonce::from_kdf).

Part	Key	Nonce	AAD
Request length	kdf16(cmd_key, [Key_Length salt, auth_id, conn_nonce])	kdf(cmd_key, [Nonce_Length salt, auth_id, conn_nonce])[..12]	auth id
Request payload	kdf16(cmd_key, [Key salt, auth_id, conn_nonce])	kdf(cmd_key, [Nonce salt, auth_id, conn_nonce])[..12]	auth id
Response length	kdf16(response BodyKey, [Resp Header Len Key salt])	kdf(response BodyIv, [Resp Header Len IV salt])[..12]	empty
Response payload	kdf16(response BodyKey, [Resp Header Key salt])	kdf(response BodyIv, [Resp Header IV salt])[..12]	empty

The salt names in the table are abbreviations of the constants listed under Salts. Each derived key and nonce pair seals exactly one message: the request pair changes with every auth id and connection nonce, and the response pair with every random body key and IV.

Typed header material

The header types are declared with byte_newtype! from protocols/src/macros.rs, which produces a Copy tuple struct with a private field and a const fn as_bytes(&self) -> &[u8; N]:

pub struct GcmKey([u8; 16]);
pub struct GcmNonce([u8; 12]);
pub struct ConnNonce([u8; 8]);

impl GcmKey {
    pub fn request_len(cmd_key: &CmdKey, auth_id: &AuthId, conn_nonce: &ConnNonce) -> Self
    pub fn request_payload(cmd_key: &CmdKey, auth_id: &AuthId, conn_nonce: &ConnNonce) -> Self
    pub fn response_len(body_key: &BodyKey) -> Self
    pub fn response_payload(body_key: &BodyKey) -> Self
}

impl GcmNonce {
    fn from_kdf(digest: [u8; 32]) -> Self
    pub fn request_len(cmd_key: &CmdKey, auth_id: &AuthId, conn_nonce: &ConnNonce) -> Self
    pub fn request_payload(cmd_key: &CmdKey, auth_id: &AuthId, conn_nonce: &ConnNonce) -> Self
    pub fn response_len(body_iv: &BodyIv) -> Self
    pub fn response_payload(body_iv: &BodyIv) -> Self
}

impl ConnNonce {
    pub fn random() -> Self
    pub const fn from_bytes(raw: [u8; 8]) -> Self
}

GcmKey and GcmNonce have no public byte constructor. Outside aead.rs the only way to obtain one is a role-named constructor, so no call site picks a salt by hand. The length and payload keys have the same type and size; their constructors, not the type, keep them apart.

Sealing and opening

pub const TAG_SIZE: usize = 16;

pub fn gcm_seal(key: &GcmKey, nonce: &GcmNonce, aad: &[u8], plaintext: &[u8]) -> Vec<u8>
pub fn gcm_open(key: &GcmKey, nonce: &GcmNonce, aad: &[u8], ciphertext: &[u8]) -> io::Result<Vec<u8>>

pub fn seal_vmess_aead_header(cmd_key: &CmdKey, data: &[u8]) -> Vec<u8>

pub async fn open_vmess_aead_header<R: AsyncRead + Unpin>(
    cmd_key: &CmdKey,
    authid: &AuthId,
    reader: &mut R,
) -> io::Result<Vec<u8>>

pub fn open_vmess_aead_header_slice(
    cmd_key: &CmdKey,
    authid: &AuthId,
    buf: &[u8],
) -> io::Result<Option<(Vec<u8>, usize)>>

Both openers expect input positioned right after the 16-byte auth id, which the caller has already consumed and authenticated. They read the 18-byte sealed length and the 8-byte connection nonce, open the length first, then read and open length + TAG_SIZE bytes of payload. The payload is never touched before the length has authenticated.

open_vmess_aead_header_slice is the sans-I/O form that VMessCore uses. It returns Ok(None) while the buffer is too short for the length part, the nonce or the full payload. Once the length part and the nonce are present, it opens the length at once, so a bad length tag is an error even before the payload has arrived. It returns Ok(Some((inner, used))) once the whole header is present, where used counts the bytes after the auth id. The core then returns used as consumed and leaves any body bytes in the buffer. The async open_vmess_aead_header over an AsyncRead is used only by unit tests.

The response header goes the other way: protocols/src/vmess/protocol.rs → encode_response_header seals the 4-byte payload [response_header, 0, 0, 0] with the response keys and an empty AAD, and the client’s decode_response_header and decode_response_header_slice open it and check that the first byte echoes the response_header value it sent.

Server handshake data flow

sequenceDiagram
  participant C as Client
  participant Core as VMessCore
  participant V as AccountValidator
  participant A as aead
  C->>Core: 16-byte auth id
  Core->>V: authenticate(authid, now)
  V-->>Core: MatchedAccount (cmd_key, uuid, user_data)
  Note over Core: state AuthId to Header, 16 bytes consumed
  C->>Core: sealed length, conn nonce, sealed payload
  Core->>A: open_vmess_aead_header_slice(cmd_key, authid, buf)
  A-->>Core: Some(inner, used)
  Note over Core: parse_request_header, chunk_streams, response_header
  Core-->>C: sealed response header (when the reply is due)

When authenticate returns None, VMessCore fails the connection with io::ErrorKind::PermissionDenied and the message vmess: unknown user or invalid auth id. The same error covers an unknown user, an id outside the time window and a replay; the core does not distinguish them.

SHAKE128

The chunk framing needs the SHAKE128 extendable-output function, and the sha3 crate at the version in use moved it to a separate crate. Rather than add a dependency, aead.rs implements FIPS 202 directly: keccak_f1600 (24 rounds with the KECCAK_RC, KECCAK_RHO and KECCAK_PI tables) and a sponge with rate SHAKE128_RATE = 168 bytes.

pub struct Shake128 {
    state: [u64; 25],
    pos: usize,
}

impl Shake128 {
    pub fn new(seed: &[u8]) -> Self
    pub fn read(&mut self, out: &mut [u8])
    pub fn next_u16(&mut self) -> u16
    pub fn next_padding_len(&mut self) -> u16
}

• new absorbs the seed in 168-byte blocks, applies the SHAKE pad10*1 padding with domain byte 0x1F and final bit 0x80, and permutes once. The instance is then squeeze-only.
• read squeezes byte by byte and permutes again each time pos reaches 168. Output does not depend on how reads are split, which the chunk framing relies on because it pulls two bytes at a time.
• next_u16 is the next two bytes, big-endian (Xray’s shakeSizeParser.next).
• next_padding_len is next_u16() % 64 (shakeSizeParser.NextPaddingLen).

protocols/src/vmess/framing.rs → ChunkStream::new seeds one instance per direction with that direction’s 16-byte BodyIv. Per chunk it draws the padding length first (only with global padding negotiated) and then the 16-bit length mask (only with chunk masking negotiated). The order matters: both ends consume the same keystream, and swapping the two draws desynchronises them from the first chunk.

Typed keys

VMess handles many unrelated 16-byte values. As bare [u8; 16] they would be interchangeable, and swapping a key for an IV compiles and fails only against a real peer. protocols/src/vmess/keys.rs gives each meaning its own type. The key_bytes! macro declares the struct and its from_bytes, as_bytes and into_bytes; secret_traits! adds a constant-time PartialEq (subtle::ConstantTimeEq) and a Debug that prints Name(<redacted>).

pub struct CmdKey([u8; 16]);
pub struct BodyKey([u8; 16]);
pub struct BodyIv([u8; 16]);
pub struct AuthId([u8; 16]);
pub struct AuthIdPlain([u8; 16]);

// key_bytes! (CmdKey, BodyKey, BodyIv, AuthId)
pub const fn from_bytes(raw: [u8; 16]) -> Self
pub const fn as_bytes(&self) -> &[u8; 16]
pub const fn into_bytes(self) -> [u8; 16]

impl BodyKey {
    pub fn random() -> Self
    pub fn response(&self) -> Self
}
impl BodyIv {
    pub fn random() -> Self
    pub fn response(&self) -> Self
}
impl AuthId {
    pub const fn shard_byte(&self) -> u8
}
impl AuthIdPlain {
    pub const fn from_bytes(raw: [u8; 16]) -> Self
    pub fn crc_ok(&self) -> bool
    pub fn timestamp(&self) -> i64
}

Type	Holds	Equality	Debug	Extras
CmdKey	MD5(uuid, magic), the root secret of an account	constant-time	redacted	none
BodyKey	Per-connection body key	constant-time	redacted	random, response = SHA-256(key)[..16]
BodyIv	Per-connection body IV	constant-time	redacted	random, response = SHA-256(iv)[..16]
AuthId	The auth-id ciphertext, as sent on the wire	plain byte compare	hex	Hash (the replay-set key), shard_byte
AuthIdPlain	The decrypted auth-id block	none	timestamp and crc_ok only	crc_ok, timestamp

The request-to-response derivation lives in BodyKey::response and BodyIv::response rather than in a free function, so a response key and a response IV can no longer be swapped at a call site. AuthIdPlain is a separate type so a decrypted block can never be passed where a ciphertext is expected. Values that exist only inside one algorithm step, such as a KDF digest fed straight into a cipher, stay as raw arrays.

AccountValidator

Key types

pub struct Account<T> {
    pub uuid: Uuid,
    pub cmd_key: CmdKey,
    pub authid_cipher: Aes128,
    pub user_data: Arc<T>,
}

impl<T> Account<T> {
    pub fn from_uuid(uuid: &Uuid, user_data: Arc<T>) -> Self
}

pub struct MatchedAccount<T> {
    pub cmd_key: CmdKey,
    pub uuid: Uuid,
    pub user_data: Arc<T>,
}

pub struct AccountValidator<T> {
    snapshot: ArcSwap<Snapshot<T>>,
    replay: [Mutex<ReplayShard>; REPLAY_SHARDS],
    deep_hits: AtomicU64,
    resort_lock: Mutex<()>,
}

impl<T> AccountValidator<T> {
    pub fn new() -> Self
    pub fn from_users(users: impl IntoIterator<Item = (Uuid, Arc<T>)>) -> Self
    pub fn add(&self, uuid: Uuid, user_data: Arc<T>)
    pub fn authenticate(&self, authid: &AuthId, now: i64) -> Option<MatchedAccount<T>>
    fn maybe_resort(&self)
}

T is the per-user payload the router sees: () in the app, a user tag in katana. The auth id cannot be indexed, since its plaintext is time || rand || crc under a per-user key, so matching is a linear trial decryption over every user. With many users that scan dominates connection setup, so the validator is built to keep it cheap: the scan itself takes no lock, and the only lock on the matching path is one replay shard, held for a single insert.

The snapshot

struct Snapshot<T> {
    deciphers: Box<[Aes128Dec]>,
    meta: Box<[UserMeta<T>]>,
    hits: Box<[AtomicU64]>,
}

struct UserMeta<T> {
    uuid: Uuid,
    cmd_key: CmdKey,
    user_data: Arc<T>,
}

impl<T> Snapshot<T> {
    fn build(users: impl IntoIterator<Item = (Account<T>, u64)>) -> Self
    fn scan(&self, authid: &AuthId, now: i64) -> Option<usize>
    fn reordered_by_hits(&self) -> Self
}

A Snapshot is immutable once published. Its three boxed slices are index-aligned and in scan order. A freshly built snapshot keeps the order the users were given in (add appends to the end); after a reorder the most-hit users come first:

• deciphers is the only array the scan reads on every probe. Keeping the decrypt-only key schedules contiguous lets a miss stream through memory sequentially instead of chasing pointers.
• meta is touched once, after a match, to copy out the cmd key, UUID and payload.
• hits counts matches per user. authenticate bumps it with Ordering::Relaxed; only a rebuild reads it.

scan returns the first index whose decipher passes decipher_matches.

authenticate

flowchart TB
  start["authenticate(authid, now)"] --> arrived["arrived = Instant::now()"]
  arrived --> load["snapshot.load()"]
  load --> scan{"scan: CRC and window"}
  scan -->|"no user"| none1["None"]
  scan -->|"index idx"| shard["shard = shard_byte AND REPLAY_MASK"]
  shard --> admit{"shard.lock().admit(authid, arrived)"}
  admit -->|"already seen"| none2["None (replay)"]
  admit -->|"new"| count["hits[idx] += 1"]
  count --> deep{"idx at least 1024 and deep_hits reaches 64"}
  deep -->|"yes"| resort["maybe_resort()"]
  deep -->|"no"| hit["Some(MatchedAccount)"]
  resort --> hit

1. Record the arrival time on the monotonic clock, before the scan.
2. Load the current snapshot through ArcSwap::load. No lock is taken, and a concurrent add or reorder cannot change the snapshot this call scans.
3. Scan. No match returns None.
4. Pick the replay shard from the auth id’s first byte and lock only that shard for one admit. A replay returns None.
5. Bump the user’s hit counter, copy out the MatchedAccount, and possibly trigger a reorder.

The scan runs before the replay check, so only ids that decrypt under a configured user’s key, pass the CRC and fall inside the window ever reach the replay set.

Replay protection

#[derive(Default)]
struct ReplayShard {
    seen: std::collections::HashSet<AuthId>,
    order: std::collections::VecDeque<(Instant, AuthId)>,
}

impl ReplayShard {
    fn admit(&mut self, authid: &AuthId, arrived: Instant) -> bool
}

The replay set is split into REPLAY_SHARDS = 64 shards, each a parking_lot::Mutex<ReplayShard>. The shard index is authid.shard_byte() as usize & REPLAY_MASK. The auth id is AES output, so its first byte is uniform and needs no further hashing; REPLAY_SHARDS must stay a power of two for the mask to work.

admit first expires the front of order: while the oldest entry’s age, measured from arrived, exceeds REPLAY_TTL, it is removed from both order and seen. admit then inserts the id into seen, returning false if it was already there, and otherwise appends (arrived, authid) to order. The timestamps are monotonic Instant values, so a wall-clock jump does not disturb the queue, and expiry from the front is amortised O(1). Expiry is lazy: a shard sheds old entries only when admit runs on it, which happens for every id that passed the scan and maps to that shard, replays included.

REPLAY_TTL is 2 * AUTHID_WINDOW_SECS = 240 s. The window is symmetric, so an id stamped 120 s in the future keeps passing the time check until the server clock reaches its stamp plus 120 s, 240 s after it first arrived. Any shorter TTL would let such an id be replayed after its entry expired. Eviction is by age only, never by count, so no volume of other traffic can push out an id that the validator admitted less than 240 s ago.

Hit-count reordering

A hit near the front of the scan is cheap; a hit deep in the list pays for every probe before it. The validator reorders the snapshot by hit count, but only when the hot set has visibly drifted out of the prefix:

Constant	Value	Role
DEEP_HIT_DEPTH	1024	A match at idx >= 1024 counts as a deep hit. Hits in the prefix never touch the reorder path.
DEEP_HITS_PER_RESORT	64	Deep hits accumulated in deep_hits before a reorder is attempted.

When a deep hit brings deep_hits to 64 or more, the thread calls maybe_resort:

1. resort_lock.try_lock(). If another thread holds it, return at once; the scan path never blocks on a reorder.
2. Re-check deep_hits under the lock, since another thread may have just reordered, then reset it to 0.
3. reordered_by_hits sorts (decipher, meta, hits) triples by descending hit count (sorted_unstable_by_key, so equal counts land in no particular order) and clones them into a new snapshot. It clones the already-expanded Aes128Dec values rather than re-running the KDF and key expansion, so a reorder costs one sort and one copy over all users, with no key derivation.
4. Publish the new snapshot with ArcSwap::store.

With 1024 users or fewer no hit is ever deep, so the order never changes. Hit counts are advisory: a match that increments a counter on a snapshot that is being replaced may not be carried into the new one.

Building the validator

Constructor	Cost	Hit counts	Used by
new()	Empty snapshot, 64 empty shards.	none	Default, add-based setup
from_users(users)	One Snapshot::build. Per user: Account::from_uuid computes cmd_key (MD5) and a full auth_id_cipher (one KDF plus AES key expansion, not kept in the snapshot), then Snapshot::build runs auth_id_decipher (a second KDF plus a decrypt key expansion).	all 0	The app (app/src/inbound/mod.rs, the "vmess" inbound arm) and katana (src/inbound.rs)
add(uuid, user_data)	Rebuilds the whole snapshot under resort_lock: O(users), re-running auth_id_cipher and auth_id_decipher for every existing user as well as the new one.	kept for existing users, 0 for the new one	Unit tests; meant for configuration time only

add takes resort_lock with a blocking lock(), so it cannot interleave with a reorder: neither can publish a snapshot that drops the other’s change. The new user is appended at the end of the scan order.

Concurrency summary

Field	Primitive	Written by	Read by
snapshot	ArcSwap<Snapshot<T>>	add, maybe_resort, both under resort_lock	authenticate, without a lock
replay	[Mutex<ReplayShard>; 64]	authenticate, one shard per call	authenticate
deep_hits	AtomicU64, Relaxed	authenticate (fetch_add), maybe_resort (reset)	authenticate, maybe_resort
resort_lock	Mutex<()>	add (lock), maybe_resort (try_lock)	none
Snapshot::hits	AtomicU64 per user, Relaxed	authenticate	reordered_by_hits, add

authenticate takes &self, so one Arc<AccountValidator<T>> is shared by every connection of an inbound.

Invariants

Invariant	Enforced by	Pinned by
The KDF’s base level is plain HMAC-SHA256 keyed with "VMess AEAD KDF".	hmac_chain at level 0	hmac_base_level_matches_reference (protocols/tests/unit/vmess/aead.rs)
An auth id round-trips through its user’s cipher with a valid CRC and the original timestamp.	create_auth_id_with_cipher, AuthIdPlain	auth_id_roundtrips_within_window (aead.rs tests)
A user matches its own fresh id.	decipher_matches	user_found_for_own_authid_within_window (protocols/tests/unit/vmess/accounts.rs)
An id made with another user’s key never matches.	AuthIdPlain::crc_ok	user_not_found_for_wrong_id (accounts.rs tests)
An id stamped outside ±120 s is rejected, in the past and in the future.	AUTHID_WINDOW_SECS in decipher_matches	user_not_found_outside_time_window (accounts.rs tests). The test uses ids 1000 s off, so the exact 120 s boundary is not pinned.
A second use of an id is rejected.	ReplayShard::admit	replayed_authid_is_rejected (accounts.rs tests); a_replayed_auth_id_is_refused (protocols/tests/unit/vmess/core.rs, expects PermissionDenied)
No volume of other traffic evicts a live replay entry.	Age-only eviction with REPLAY_TTL	replay_survives_heavy_traffic (accounts.rs tests, 20 000 distinct ids)
Entries older than REPLAY_TTL leave the set.	Front-of-queue expiry in admit	expired_ids_leave_the_set (accounts.rs tests). The test drives expire_all, a test-only copy of the expiry loop, not admit itself.
A hot user deep in the list is hoisted to the front, and every other user stays matchable.	deep_hits, maybe_resort, reordered_by_hits	deep_hits_hoist_the_hot_user_into_the_prefix (accounts.rs tests)
A user added after construction is matchable.	add rebuilds and publishes the snapshot	add_after_construction_is_matchable (accounts.rs tests)
A sealed header opens to the original bytes, from a stream and from a slice.	seal_vmess_aead_header, both openers	Stream: header_seal_open_roundtrips (aead.rs tests), request_header_roundtrips (protocols/tests/unit/vmess/protocol.rs). Slice: request_and_response_headers_open_from_slices (protocol.rs tests), stream_codec_seals_the_header_and_chunks_and_opens_the_response (protocols/tests/unit/vmess/codec.rs)
The slice opener returns None, not an error, while the header is incomplete, and reports exactly the bytes it used.	open_vmess_aead_header_slice	request_and_response_headers_open_from_slices (protocol.rs tests)
The response header seals and opens with the response keys, and the decoder accepts it only when the first byte echoes response_header.	encode_response_header, decode_response_header, decode_response_header_slice	response_header_roundtrips (protocol.rs tests, fixed keys); request_and_response_headers_open_from_slices (keys from RequestSession::response). No test feeds a wrong echo byte.
Shake128 matches FIPS 202, and split reads equal one bulk read.	keccak_f1600, sponge padding	shake128_fips202_empty, shake128_streaming_matches_bulk (aead.rs tests)
All salts, the cmd-key magic and the SHAKE usage match Xray.	Constants copied from Xray	app_server_vmess_grpc_xray_client_tls, app_client_vmess_ws_xray_server_early_data_plain, app_server_vmess_ws_xray_client_early_data_plain (app/tests/integration/e2e_xray_vmess.rs). They build Xray from source with go and return early, passing, when go is missing or the build fails.
Keys, IVs, auth ids and decrypted blocks cannot be substituted for each other.	Distinct newtypes in keys.rs; GcmKey and GcmNonce have no public byte constructor	The compiler; no runtime test

The server core and the client codecs are also run against each other over real sockets in protocols/tests/pipeline/vmess.rs. new_server_vs_new_client_tcp runs AES-128-GCM with global padding and ChaCha20-Poly1305 without it; new_server_vs_new_client_udp runs AES-128-GCM with global padding.

Failure paths

Condition	Where	Result
Fewer than 16 bytes buffered	VMessCore, State::AuthId	Consumes 0 and waits for more.
No user matches, id out of window, or replay	AccountValidator::authenticate returns None	VMessCore fails with PermissionDenied, vmess: unknown user or invalid auth id.
Header not yet complete	open_vmess_aead_header_slice returns Ok(None)	Consumes 0 and waits for more.
Length or payload tag does not verify	gcm_open	InvalidData, vmess: AEAD header open failed.
Length arithmetic overflows	ProtocolError::Overflow("aead header length")	InvalidData, integer overflow: aead header length. Unreachable in practice, since the length is a u16.
Stream ends inside the header	open_vmess_aead_header (read_exact)	UnexpectedEof.
Response header tag fails, or its first byte differs	decode_response_header, decode_response_header_slice	InvalidData (vmess: AEAD header open failed or vmess: unexpected response header).

gcm_seal cannot fail for header-sized inputs; if the AEAD library ever returned an error it yields an empty buffer rather than panicking. The code avoids panics on peer input by construction: every slice access that depends on received bytes or a received length goes through take_array or get, and arithmetic on peer-supplied lengths is checked. The remaining fixed-offset indexing is on arrays of known size. No test feeds malformed bytes to these functions specifically.

A failed handshake has no cancellation concerns of its own: every function on this page is synchronous except open_vmess_aead_header, which only awaits read_exact, and the handshake deadline belongs to the core’s timing (see Server core).

Limits

Name	Value	Where
TAG_SIZE	16 bytes	aead.rs; GCM and ChaCha20-Poly1305 tags
Auth id	16 bytes	AuthId
ConnNonce	8 bytes	aead.rs
Sealed length field	18 bytes (2 + TAG_SIZE)	open_vmess_aead_header_slice → LEN_PART
Inner header length	up to 65535 bytes (u16)	sealed length field
SHAKE128_RATE	168 bytes	aead.rs
next_padding_len	0 to 63	aead.rs
AUTHID_WINDOW_SECS	120 s, each direction	accounts.rs
REPLAY_TTL	240 s	accounts.rs
REPLAY_SHARDS	64	accounts.rs
REPLAY_MASK	63	accounts.rs
DEEP_HIT_DEPTH	1024	accounts.rs
DEEP_HITS_PER_RESORT	64	accounts.rs

Tests

Test	File	Pins
shake128_fips202_empty	protocols/tests/unit/vmess/aead.rs	First 32 output bytes of SHAKE128("") match FIPS 202.
shake128_streaming_matches_bulk	same	48 two-byte reads equal one 96-byte read.
hmac_base_level_matches_reference	same	KDF level 0 equals hmac::SimpleHmac<Sha256>.
auth_id_roundtrips_within_window	same	CRC and timestamp survive encrypt and decrypt.
header_seal_open_roundtrips	same	Seal, split off the auth id, open with the async opener.
user_found_for_own_authid_within_window	protocols/tests/unit/vmess/accounts.rs	Match returns the right cmd key, UUID and payload.
add_after_construction_is_matchable	same	new then add produces a matchable user.
user_not_found_outside_time_window	same	Ids 1000 s in the past and future are rejected.
user_not_found_for_wrong_id	same	Another user’s id is rejected.
replayed_authid_is_rejected	same	Second use of an id returns None.
replay_survives_heavy_traffic	same	20 000 other ids do not evict a live entry.
expired_ids_leave_the_set	same	Entries past REPLAY_TTL are removed.
deep_hits_hoist_the_hot_user_into_the_prefix	same	1280 users; 65 hits on the last one move it to index 0.
request_header_roundtrips	protocols/tests/unit/vmess/protocol.rs	Full request header through seal and async open.
response_header_roundtrips	same	Response header seal and open.
request_and_response_headers_open_from_slices	same	Slice openers on truncated and complete buffers.
stream_codec_seals_the_header_and_chunks_and_opens_the_response	protocols/tests/unit/vmess/codec.rs	A client codec’s header opens with open_vmess_aead_header_slice, and used points at the first body chunk.
a_replayed_auth_id_is_refused	protocols/tests/unit/vmess/core.rs	Two cores sharing a validator; the replay fails with PermissionDenied.
new_server_vs_new_client_tcp, new_server_vs_new_client_udp	protocols/tests/pipeline/vmess.rs	Server core against client codecs over sockets.
app_server_vmess_grpc_xray_client_tls and the two WebSocket early-data tests	app/tests/integration/e2e_xray_vmess.rs	Byte compatibility with a real Xray binary.

The accounts.rs tests use test-only helpers on AccountValidator (user_count, scan_depth, seen_len, expire_all) and auth_id_with_nonce, which builds an id with a chosen random field so tests can create many distinct, valid ids in the same second. Run the unit tests with cargo test -p etemenanki-protocols vmess.

Related

VMess: wire format and coreInner header layout, chunk framing and the VMessCore state machine.

VMess user guideConfiguring VMess inbounds and outbounds.

Protocol foundationsShared helpers, errors and parsing rules used by every protocol.

Server coreThe sans-I/O core model that VMessCore implements.

Shadowsocks 2022Another AEAD protocol with a timestamp window.

TestingHow unit, pipeline and interoperability tests are organised.