Validation and apply errors
Source files: 44 · checked against Etemenanki 555b7df · katana v4.1.1
Etemenanki/supervisor/src/build/validate.rsEtemenanki/supervisor/src/build/apply.rsEtemenanki/supervisor/src/build/mod.rsEtemenanki/supervisor/src/lib.rsEtemenanki/supervisor/src/build/users.rsEtemenanki/supervisor/src/build/inbound.rsEtemenanki/supervisor/src/build/outbound.rsEtemenanki/supervisor/src/entity/id.rsEtemenanki/supervisor/src/entity/user.rsEtemenanki/supervisor/src/supervisor.rsEtemenanki/supervisor/src/topology/spec_plan/plan.rsEtemenanki/supervisor/src/topology/spec_plan/inbound.rsEtemenanki/supervisor/src/topology/spec_plan/transport.rsEtemenanki/supervisor/src/topology/inbound/mod.rsEtemenanki/supervisor/src/topology/balancer.rsEtemenanki/supervisor/src/topology/plane.rsEtemenanki/supervisor/src/system/listener.rsEtemenanki/protocols/src/hysteria/server/masquerade.rsEtemenanki/protocols/src/hysteria/auth.rsEtemenanki/protocols/src/hysteria/obfs.rsEtemenanki/protocols/src/tun/device.rsEtemenanki/protocols/src/ss_2022/users.rsEtemenanki/protocols/src/ss_2022/crypto.rsEtemenanki/protocols/src/helpers/address_family.rsEtemenanki/app/src/instance.rsEtemenanki/app/src/lower.rsEtemenanki/app/src/main.rsEtemenanki/app/src/api.rsEtemenanki/ffi/src/error.rsEtemenanki/webclient/src/lib.rsEtemenanki/webclient/src/router.rsEtemenanki/supervisor/tests/unit/validate.rsEtemenanki/supervisor/tests/unit/plan.rsEtemenanki/supervisor/tests/hot_swap.rsEtemenanki/protocols/tests/unit/hysteria/server/masquerade.rsEtemenanki/protocols/tests/unit/ss_2022/users.rsEtemenanki/app/tests/unit/lower.rsEtemenanki/app/tests/unit/subscribe.rskatana/src/main.rskatana/src/runtime.rskatana/src/manager/node.rskatana/src/lower/inbound.rskatana/src/lower/mod.rskatana/tests/unit/lower/outbound.rs
A Spec<U> is typed, but types cannot say which fields go together, which tags exist or which values are in range. Those rules live in one function, supervisor/src/build/validate.rs → validate, and every path that runs a spec goes through it before anything is built. A spec that breaks a rule is refused whole with an ApplyError, and what was running keeps running. This page lists every rule in the order validate checks it, with the variant and the exact text it produces, then covers the other ApplyError variants, the dry run check(), and how each front end reports the errors.
It is for contributors who add a field to a spec type, change a rule, or write a front end that lowers its own format into a spec. The spec types themselves are on The spec: desired state. What happens after validation (the plan, prepare and commit) is on Planning and applying a change.
Responsibilities
Section titled “Responsibilities”build/validate.rs owns every rule about what a spec means. Its module comment states the split: a front end owns only syntax, which is parsing its own format and reading files, so a rule is never checked on one side only. The module is pure: no sockets, no files, no clock. Validation never parses PEM either; certificate bytes are opaque to it, and the unit tests pass b"cert" as a certificate.
One semantic rule is still enforced by a protocol builder instead: a Shadowsocks 2022 inbound on 2022-blake3-chacha20-poly1305 that admits at least one user passes validate, and protocols/src/ss_2022/users.rs → Validator::from_config refuses it while the supervisor prepares the apply, as ApplyError::Build (see Build). A change to that rule has to be made there.
validate checks:
- that tags are non-empty and unique within their kind, and that no balancer shares a tag with an outbound;
- that every tag a balancer, a route rule, the route default or an inbound names exists;
- that every balancer member can be probed over TCP;
- each outbound’s fields against each other (TLS settings against the shape, key lengths, WireGuard address families);
- that no two inbounds share a bind, and that each protocol sits on its kind of bind;
- each inbound’s fields against each other (TLS material, open mode, Hysteria 2 settings, TUN MTU and platform);
- the users each inbound would admit (
validate_admission).
It does not:
- read certificates, keys, CA bundles or geodata. The builders in prepare parse and load those (
compile_routesreads geodata), and a failure isApplyError::Build. - bind anything. A port in use fails as
ApplyError::Bind. - decide defaults or fill gaps. An outbound’s WebSocket host, gRPC authority and TLS name are the front end’s to fill; a gap that reaches the supervisor is refused rather than guessed at (see
an_outbound_needs_a_ws_host_and_a_grpc_authority). - collect errors. It returns the first broken rule, so one refusal reports one problem.
Inbounds and outbounds share one StreamShape (supervisor/src/topology/spec_plan/transport.rs) and the TLS pairing helper tls_mismatch, so the TLS rule is written once for both sides. The WebSocket host and gRPC authority rules apply to outbounds only.
Where validation runs
Section titled “Where validation runs”| Caller | What runs | When |
|---|---|---|
topology/spec_plan/plan.rs → plan |
validate(desired), as its first statement |
Every Supervisor::apply, apply_with, update, update_with, SupervisorBuilder::start and check |
supervisor.rs → Actor::edit_users |
validate_admission for each inbound admitting the edited set |
Supervisor::set_users, upsert_user and remove_user |
supervisor.rs → Actor::prepare |
probe_target |
Building a balancer, to find each member’s probe destination |
Because plan validates before it computes a single step, no code path reaches the builders with an unvalidated spec. The builders rely on it: build/mod.rs says every builder takes a validated spec, and Actor::prepare has expect("validated: members are outbounds") and expect("validated: members are probeable") where it builds a balancer.
Key types
Section titled “Key types”The two entry points
Section titled “The two entry points”/// Check every semantic rule of `spec`.pub fn validate<U: UserId>(spec: &Spec<U>) -> Result<(), ApplyError>;
/// Check the users `inbound` would admit from `set`: no credential with two/// owners, and every credential usable by the inbound's protocol.pub fn validate_admission<U: UserId>( inbound: &InboundSpec, set: &UserSet<U>,) -> Result<(), ApplyError>;
/// Where a TCP health probe reaches `outbound`, if anywhere.pub(crate) fn probe_target(outbound: &OutboundSpec) -> Option<&Destination>;Both public functions are reachable as etemenanki_supervisor::build::validate::validate and …::validate_admission, so a front end can run them without a supervisor. Of the modules under build/, only apply and validate are pub; dns, inbound, outbound, route and users are pub(crate). The private helpers are:
| Helper | Returns | Does |
|---|---|---|
unique_tags(kind, tags) |
Result<HashSet<&CompactString>, ApplyError> |
Refuses an empty tag, then a repeated one, in iteration order; returns the set for the reference checks |
validate_outbound(outbound) |
Result<(), ApplyError> |
The per-outbound rules |
outbound_transport(transport) |
Result<(), CompactString> |
TLS pairing, then the WebSocket host and gRPC authority |
tls_mismatch(shape) |
CompactString |
The text for a TLS pairing error, which depends on which side is missing |
obfs(obfs) |
Result<(), CompactString> |
The Salamander key length, shared by inbound and outbound |
validate_inbound(inbound) |
Result<(), ApplyError> |
The per-inbound rules |
invalid_inbound(inbound, reason) |
ApplyError |
ApplyError::Invalid naming Resource::Inbound(tag) |
Inside validate, the sets that unique_tags returns do the reference checks. A balancer’s members are looked up in by_tag, a HashMap<&CompactString, &OutboundSpec> over the outbounds, and each route target goes through the closure routable, which is true for a tag in the outbound set or the balancer set. validate_admission keeps the identities it has seen in seen, a HashMap<Vec<u8>, &U> from identity to the user who presented it first.
Constants
Section titled “Constants”| Constant | Type | Value | Rule |
|---|---|---|---|
MIN_TUN_MTU |
u16 |
1280 |
The smallest MTU the userspace stack serves |
HY2_UDP_IDLE |
RangeInclusive<u64> |
2..=600 (seconds, inclusive) |
A Hysteria 2 UDP association’s idle timeout. The comment gives the reason: below it a busy association is swept between packets, above it a dead one holds its socket for ten minutes |
MIN_SALAMANDER_PSK |
usize |
4 |
The shortest Salamander key, on either side. protocols/src/hysteria/obfs.rs → MIN_PSK_LEN is also 4, and Salamander::new refuses the same keys with ProtocolError::Malformed("hysteria2 obfs psk is too short") |
Rules also read these values from etemenanki-protocols:
| Value | Source | Used for |
|---|---|---|
STATUS_AUTH_OK = 233 |
protocols/src/hysteria/auth.rs |
The one status a masquerade may not use |
Method::key_len() = 16 for 2022-blake3-aes-128-gcm, 32 for 2022-blake3-aes-256-gcm and 2022-blake3-chacha20-poly1305 |
protocols/src/ss_2022/crypto.rs |
Shadowsocks 2022 key sizes |
normalise_psk(method, psk) |
protocols/src/ss_2022/users.rs |
A user PSK: accepted at the method’s length, folded to it when longer (the first key_len bytes of its SHA-256, ss_2022/crypto.rs → fold_key), refused when shorter with ErrorKind::InvalidInput and shadowsocks-2022: PSK too short (<len> < <n>) |
The spec carries Shadowsocks 2022 keys as decoded bytes. Front ends decode the base64 text of a key with ss_2022/users.rs → decode_psk, which fails with ErrorKind::InvalidInput and decode PSK: <e>; that is a lowering error, never an ApplyError.
ApplyError
Section titled “ApplyError”/// Why a spec was not applied.////// A spec that fails validation is refused as a whole, and what was running/// keeps running.#[derive(Debug, thiserror::Error)]pub enum ApplyError { /// Two resources of one kind share a tag, or a balancer's tag is also an /// outbound's. #[error("duplicate {kind} tag {tag}")] DuplicateTag { kind: ResourceKind, tag: CompactString }, /// `from` names a tag nothing of that kind carries: a rule or a balancer /// naming a missing outbound, or an inbound naming a missing user set. #[error("{from} references unknown {kind} {tag}")] UnknownReference { from: Resource, kind: ResourceKind, tag: CompactString }, /// A balancer with nothing to choose from. #[error("balancer {balancer} has no members")] EmptyBalancer { balancer: CompactString }, /// A balancer member no TCP health probe can reach, so its health, which /// is all a balancer selects on, could never be known. #[error("balancer {balancer}: outbound {member} has no upstream a TCP health probe can reach")] UnprobeableMember { balancer: CompactString, member: CompactString }, /// A spec that breaks one of its documented invariants, such as TLS /// material missing under a shape that layers TLS, or a Trojan inbound /// without a user set. #[error("{resource}: {reason}")] Invalid { resource: Resource, reason: CompactString }, /// A resource's spec was valid, but constructing it failed: a /// certificate that does not parse, geo data that cannot be read. #[error("building {resource} failed: {source}")] Build { resource: Resource, #[source] source: io::Error }, /// An inbound's listener could not be bound. #[error("inbound {inbound}: binding {bind} failed: {source}")] Bind { inbound: CompactString, bind: BindSpec, #[source] source: io::Error }, /// The spec needs a change that ends live connections, and the apply /// did not allow one. #[error("inbound {inbound}: {reason}; this ends its live connections and needs allow_disruptive")] Disruptive { inbound: CompactString, reason: CompactString }, /// The supervisor has shut down. #[error("the supervisor has shut down")] Stopped,}The enum doc states the contract: a spec that fails validation is refused as a whole, and what was running keeps running. lib.rs re-exports the module at the crate root (pub use build::apply;), so etemenanki_supervisor::apply::ApplyError and etemenanki_supervisor::apply::ApplyReport are a second path to the same types; etemenanki-app imports them from build::apply.
| Variant | Stage | Produced by |
|---|---|---|
DuplicateTag |
validate | unique_tags, and the balancer and outbound collision check |
UnknownReference |
validate | A balancer member, a route rule or the route default, or an inbound’s user set naming a tag that does not exist |
EmptyBalancer |
validate | A balancer with no members |
UnprobeableMember |
validate | A balancer member for which probe_target returns None |
Invalid |
validate, user edits | Every other rule of validate and validate_admission; also edit_users for an unknown set |
Build |
prepare | A builder or a listener preparation step that failed. Validation passed |
Bind |
prepare | system/listener.rs → bind for a new listener |
Disruptive |
after plan, before prepare | A plan with a Step::Disrupt and ApplyOptions::allow_disruptive unset |
Stopped |
any | The actor is gone, or there is no running spec to edit |
Build and Bind carry the io::Error both in their text and as source(). ApplyError does not implement Clone.
How resources print
Section titled “How resources print”Every text above builds on two Display implementations in supervisor/src/entity/id.rs:
| Value | Prints |
|---|---|
ResourceKind::Inbound / Outbound / Balancer / UserSet |
inbound / outbound / balancer / user set |
Resource::Inbound(tag) |
inbound <tag> |
Resource::Outbound(OutboundId) |
outbound <tag>@v<version> |
Resource::Balancer(tag) |
balancer <tag> |
Resource::UserSet(tag) |
user set <tag> |
Resource::Route |
route |
Resource::Dns |
dns |
A bind prints through topology/inbound/mod.rs → impl Display for BindSpec:
BindSpec |
Prints |
|---|---|
Tcp { host, port } |
<host>:<port> |
Udp { host, port } |
udp <host>:<port> |
Unix(path) |
unix:<path> |
Tun(TunSource::Create(spec)) |
tun <name>, or tun auto when name is None |
Tun(TunSource::Fd(device)) |
tun fd <n> |
ApplyReport
Section titled “ApplyReport”A successful apply returns what it did to each resource. The type sits beside ApplyError in build/apply.rs:
#[derive(Debug, Clone, Default, PartialEq, Eq)]pub struct ApplyReport { /// Kept as they were: their spec did not change. pub reused: Vec<Resource>, /// Constructed from their spec: new, or changed, in which case an /// outbound is a new version of its tag. pub built: Vec<Resource>, /// Inbounds that kept their listener and now serve new connections with /// a new handler. pub swapped: Vec<CompactString>, /// Outbound versions taken out of service. Their live flows are handled /// by the drain policy. pub drained: Vec<OutboundId>, /// Inbounds whose listener was closed and bound again, because their /// BindSpec changed. pub rebound: Vec<CompactString>, /// Inbounds whose runtime was stopped and started again on the handle it /// kept, ending its live connections: the disruptive changes an apply /// was allowed to make. pub restarted: Vec<CompactString>, /// Inbounds taken out of the spec: their listeners closed, and their /// sessions with them. pub removed: Vec<CompactString>,}Every resource of the desired spec appears in reused or built; the other five lists say what that meant for what was already running. It has no Display. Planning and applying a change says how each list is filled. etemenanki-app prints it with app/src/instance.rs → summary: each non-empty list as <label> [<item>, …], joined by ; , in the order built, swapped, drained, rebound, restarted, removed, reused, or nothing to run when all are empty. Items print with the Display forms above, and drained outbounds as <tag>@v<version>.
/// Validate `spec` and build everything it describes — outbounds, routes/// with their geo data, handlers with their certificates, user tables —/// without binding a listener or starting anything. It refuses what a start/// would, short of a port or device that cannot be bound.pub async fn check<U: UserId>(spec: &Spec<U>) -> Result<(), ApplyError>;It is re-exported at the crate root as etemenanki_supervisor::check. See The dry run.
Data flow
Section titled “Data flow”Where each variant can come from during one apply:
flowchart LR FE["Supervisor::apply"] -->|"Command::Apply"| PLAN["plan: validate first"] FE -->|"actor gone"| ST["Stopped"] PLAN -->|"a rule is broken"| VE["DuplicateTag, UnknownReference, EmptyBalancer, UnprobeableMember, Invalid"] PLAN --> DIS["plan.disruptions()"] DIS -->|"allow_disruptive unset"| DE["Disruptive"] DIS --> PREP["prepare, bind = true"] PREP -->|"construction fails"| BE["Build"] PREP -->|"a listener cannot bind"| BI["Bind"] PREP --> COM["commit: cannot fail"]
The order inside validate. Each box stops at its first broken rule:
flowchart TB T1["user set tags"] --> T2["outbound tags"] T2 --> T3["balancer tags"] T3 --> T4["balancer tag equal to an outbound tag"] T4 --> T5["inbound tags"] T5 --> O["validate_outbound, each outbound in order"] O --> B["each balancer: members exist and are probeable"] B --> R["route rules in order, then the default"] R --> I1["next inbound: bind unique, user set exists"] I1 --> I2["validate_inbound"] I2 --> I3["validate_admission, when it names a set"] I3 -->|"more inbounds"| I1
The inbound checks run per inbound, so inbound 2’s admission is checked before inbound 3’s bind. Within one list, the first offending item in spec order is the one reported, with one exception: the balancer and outbound collision check scans balancers, the HashSet that unique_tags returned, so when several balancer tags equal outbound tags, the one reported is whichever the set yields first.
The rules, in check order
Section titled “The rules, in check order”The texts below were captured from the pinned etemenanki-app with --test, where the app could produce them, and read from the code otherwise. A placeholder in angle brackets stands for a value. The unit tests in supervisor/tests/unit/validate.rs match on the variant and the resource, never on the text.
1. Tags
Section titled “1. Tags”unique_tags runs once per kind, in the order user sets, outbounds, balancers, inbounds, with the balancer and outbound collision check between balancers and inbounds. For each tag it checks emptiness first, then repetition.
| Rule | Variant | Text | Test |
|---|---|---|---|
| A user set tag is not empty | Invalid { resource: UserSet("") } |
user set : a user set tag must not be empty |
no_tag_may_be_empty |
| User set tags are unique | DuplicateTag { kind: UserSet } |
duplicate user set tag <tag> |
tags_are_unique_within_each_kind |
| An outbound tag is not empty | Invalid { resource: Outbound } |
outbound @v0: a outbound tag must not be empty |
no_tag_may_be_empty |
| Outbound tags are unique | DuplicateTag { kind: Outbound } |
duplicate outbound tag <tag> |
tags_are_unique_within_each_kind |
| A balancer tag is not empty | Invalid { resource: Balancer("") } |
balancer : a balancer tag must not be empty |
none |
| Balancer tags are unique | DuplicateTag { kind: Balancer } |
duplicate balancer tag <tag> |
tags_are_unique_within_each_kind |
| No balancer shares a tag with an outbound | DuplicateTag { kind: Balancer } |
duplicate balancer tag <tag> |
a_balancer_may_not_share_an_outbounds_tag |
| An inbound tag is not empty | Invalid { resource: Inbound("") } |
inbound : a inbound tag must not be empty |
no_tag_may_be_empty |
| Inbound tags are unique | DuplicateTag { kind: Inbound } |
duplicate inbound tag <tag> |
tags_are_unique_within_each_kind |
Why the rules are what they are:
- Outbound and balancer tags share one namespace. A route names either kind by tag, so one tag on both would be ambiguous. Inbound tags and user set tags live apart:
an_inbound_may_share_an_outbounds_tagaccepts an inbound taggeddirectbeside an outbound taggeddirect. - The empty tag is reserved. The supervisor makes two targets of its own and tags both with the empty string: the blackhole of
topology/plane.rs→Plane::empty, the plane an actor holds before its first apply publishes one, and the DNS service reached as an outbound (supervisor.rs→internal_id). Because no spec tag is empty, neither names one of the spec’s outbounds. - Checks run in kind order, not file order. etemenanki-app lowers the users listed inside an inbound to a user set tagged with the inbound’s own tag. Two Trojan inbounds both tagged
tare therefore refused asduplicate user set tag t, notduplicate inbound tag t, and a Trojan inbound with an empty tag asuser set : a user set tag must not be empty. Both texts were captured from the app.
2. Outbounds
Section titled “2. Outbounds”validate_outbound runs for each outbound in spec order. Every error it returns is ApplyError::Invalid naming the outbound (Resource::Outbound).
For the proxy protocols that dial over a transport (SOCKS, HTTP, Trojan, VLESS, VMess, Shadowsocks and Shadowsocks 2022), outbound_transport checks the upstream’s OutboundTransportSpec first:
| Rule | Text | Test |
|---|---|---|
| The shape layers TLS, so TLS settings are present | outbound <tag>@v0: the transport layers tls but carries no tls settings |
an_outbound_carries_tls_settings_exactly_when_its_shape_layers_tls |
| TLS settings are present, so the shape layers TLS | outbound <tag>@v0: the transport carries tls settings but does not layer tls |
same |
A WebSocket shape has a host |
outbound <tag>@v0: ws transport needs a host |
an_outbound_needs_a_ws_host_and_a_grpc_authority |
A gRPC shape has an authority |
outbound <tag>@v0: grpc transport needs an authority |
same |
StreamShape::uses_tls decides the pairing: Tcp never layers TLS, Tls always does, and Ws and Grpc do when their tls flag is set.
Then the protocol’s own rules:
| Protocol | Rule | Text | Test |
|---|---|---|---|
| Shadowsocks 2022 | psk and every entry of identity_psks are exactly method.key_len() bytes |
outbound <tag>@v0: shadowsocks-2022 keys must be <n> bytes for this method |
shadowsocks_2022_client_keys_are_sized_to_their_method |
| Hysteria 2 | password is not empty |
outbound <tag>@v0: hysteria2 password must not be empty |
a_hysteria2_client_needs_a_password_and_a_stream |
| Hysteria 2 | max_concurrent_streams is at least 1 |
outbound <tag>@v0: max_concurrent_streams must be at least 1 |
same |
| Hysteria 2 | a Salamander key is at least 4 bytes | outbound <tag>@v0: salamander obfs key must be at least 4 bytes |
a_salamander_key_is_at_least_4_bytes_on_either_side |
| WireGuard | address_family = Ipv4Only needs an IPv4 address among addresses |
outbound <tag>@v0: wireguard address_family ipv4_only needs an IPv4 address |
a_wireguard_address_family_needs_an_address_of_that_family |
| WireGuard | address_family = Ipv6Only needs an IPv6 address |
outbound <tag>@v0: wireguard address_family ipv6_only needs an IPv6 address |
same |
The family name in the WireGuard text comes from AddressFamilyStrategy::as_str. Auto, PreferIpv4 and PreferIpv6 need no particular address. Freedom and blackhole outbounds have no rules here.
Captured from the app:
configuration invalid: outbound wg@v0: wireguard address_family ipv6_only needs an IPv6 addressconfiguration invalid: outbound hy2-out@v0: hysteria2 password must not be emptyconfiguration invalid: outbound hy2-out@v0: max_concurrent_streams must be at least 1configuration invalid: outbound hy2-out@v0: salamander obfs key must be at least 4 bytes3. Balancers
Section titled “3. Balancers”For each balancer in spec order:
| Rule | Variant | Text | Test |
|---|---|---|---|
| It has at least one member | EmptyBalancer |
balancer <b> has no members |
a_balancer_needs_a_member |
| Each member, in order, is an outbound tag | UnknownReference { from: Balancer(b), kind: Outbound } |
balancer <b> references unknown outbound <m> |
a_balancer_member_must_be_a_known_outbound |
| Each member can be probed over TCP | UnprobeableMember |
balancer <b>: outbound <m> has no upstream a TCP health probe can reach |
a_balancer_member_needs_an_upstream_a_tcp_probe_can_reach |
Members are looked up among outbounds only, so a balancer naming another balancer fails the second rule: balancer outer references unknown outbound inner (a_balancer_member_may_not_be_another_balancer). Balancers are members of nothing.
probe_target decides the third rule. Health is all a balancer selects on, and a TCP connect is how it learns health, so a member without a TCP upstream would be marked down forever:
| Outbound protocol | probe_target |
|---|---|
| SOCKS, HTTP, Trojan, VLESS, VMess, Shadowsocks, Shadowsocks 2022 | Some(&upstream.server) |
| Freedom, blackhole | None: no upstream |
| Hysteria 2, WireGuard | None: the upstream listens on UDP |
Actor::prepare calls probe_target again for each member and hands the destination to Member::new. Balancer::new still refuses an empty member list with a balancer needs at least one outbound (as a Build error), but validation refuses that spec first.
4. The route
Section titled “4. The route”Each rule’s outbound, in rule order, and then route.default must name an outbound or a balancer:
| Rule | Variant | Text | Test |
|---|---|---|---|
| A rule names a known outbound or balancer | UnknownReference { from: Route, kind: Outbound } |
route references unknown outbound <tag> |
a_route_rule_must_name_a_known_outbound_or_balancer |
| The default names a known outbound or balancer | same | same | the_route_default_must_name_a_known_outbound_or_balancer |
The text says outbound even when a balancer was meant; the kind is fixed because both share one namespace. a_route_may_name_a_balancer accepts a default that names a balancer. Whether geosite and geoip codes exist is not validation’s to say: geodata is read by build/route.rs → compile_routes, so a missing or unreadable file, or a missing code, is a Build error on route.
5. Inbounds
Section titled “5. Inbounds”For each inbound in spec order, four steps:
-
The bind is unique. Its
BindSpecis compared with those of the inbounds before it (aVec<&BindSpec>collected as the loop goes). Equal values fail asInvalid { resource: Inbound(<the later inbound>) }with<bind> is bound by another inbound. Captured:inbound b: 127.0.0.1:1080 is bound by another inbound. TCP and UDP binds are different kinds, so a Hysteria 2 inbound may use a SOCKS inbound’s port number (the_same_port_over_udp_is_another_bind). Test:two_inbounds_may_not_share_a_bind.“Equal” is
BindSpec’sPartialEq:BindSpecEqual when Tcp { host, port },Udp { host, port }Same kind, the same hoststring compared as text, and the same portUnix(path)The PathBufs are equal, which compares components: a doubled/and an inner.collapse,..does notTun(TunSource::Create(spec))The DeviceSpecs are equal field by fieldTun(TunSource::Fd(device))topology/inbound/mod.rs→impl PartialEq for SuppliedTun: the same descriptor number and the same MTU -
The user set exists. When
usersisSome(tag)and no user set carries that tag:UnknownReference { from: Inbound(tag), kind: UserSet }, printedinbound <tag> references unknown user set <set>. Test:an_inbound_must_name_a_known_user_set. -
validate_inbound, below. -
validate_admissionagainst the named set, when there is one. See Admission.
validate_inbound
Section titled “validate_inbound”Every error is Invalid { resource: Inbound(tag) }, printed inbound <tag>: <reason>.
The pairing comes first. Each protocol is served on one kind of bind:
| Protocol | Bind it needs |
|---|---|
| SOCKS, HTTP, Trojan, VLESS, VMess, Shadowsocks, Shadowsocks 2022 | BindSpec::Tcp or BindSpec::Unix |
| Hysteria 2 | BindSpec::Udp |
| TUN | BindSpec::Tun |
Any other combination fails with the protocol cannot be served on <bind>, for example inbound in: the protocol cannot be served on udp 127.0.0.1:2000. Test: each_protocol_is_served_only_on_its_kind_of_bind.
For the protocols that carry an InboundTransportSpec (HTTP, Trojan, VLESS, VMess), a transport check runs next, in this order:
| Rule | Reason text | Test |
|---|---|---|
On a Unix bind, the shape is StreamShape::Tcp |
a unix socket carries no transport; its shape must be plain tcp |
a_unix_listener_carries_only_the_plain_tcp_shape |
| The shape layers TLS, so TLS material is present | the transport layers tls but carries no tls settings |
an_inbound_carries_tls_material_exactly_when_its_shape_layers_tls |
| TLS material is present, so the shape layers TLS | the transport carries tls settings but does not layer tls |
same |
An inbound’s WebSocket host and gRPC authority may be None: a server has nothing to borrow them from and needs neither (an_inbound_needs_no_ws_host_or_grpc_authority).
Then the protocol’s own rules:
| Protocol | Rule | Reason text | Test |
|---|---|---|---|
| SOCKS | On a Unix bind with udp on, udp_bind is set |
socks over a unix socket has no local IP for UDP associate; set udp_bind or turn udp off |
socks_over_a_unix_socket_needs_a_udp_bind_to_serve_udp |
| Trojan, VLESS, VMess | users is Some |
the protocol has no open mode and needs a user set |
trojan_vless_and_vmess_need_a_user_set |
| Shadowsocks | none | ||
| Shadowsocks 2022 | The server psk is exactly method.key_len() bytes |
the shadowsocks-2022 key must be <n> bytes for this method |
a_shadowsocks_2022_server_key_is_sized_to_its_method |
| Hysteria 2 | Not both shared_password and users |
a shared password and a user set cannot both be set; a credential would have two answers |
hysteria2_takes_a_shared_password_or_a_user_set_but_not_both |
| Hysteria 2 | One of them is set | hysteria2 needs a shared password or a user set |
same |
| Hysteria 2 | A shared password is not empty | the shared password must not be empty |
a_hysteria2_shared_password_may_not_be_empty |
| Hysteria 2 | udp_idle_timeout, when set, is 2 to 600 whole seconds |
udp_idle_timeout must be between 2 and 600 seconds |
a_hysteria2_udp_idle_timeout_is_between_2_and_600_seconds |
| Hysteria 2 | max_connections and max_circuits are at least 1 |
max_connections and max_circuits must be at least 1 |
hysteria2_serves_at_least_one_connection_and_one_circuit |
| Hysteria 2 | A Salamander key is at least 4 bytes | salamander obfs key must be at least 4 bytes |
a_salamander_key_is_at_least_4_bytes_on_either_side |
| Hysteria 2 | A masquerade builds with Masquerade::new |
the constructor’s error text | a_masquerade_may_not_answer_with_the_success_status |
| TUN | The device MTU (TunSource::mtu) is at least 1280 |
tun mtu must be at least 1280 |
a_tun_device_mtu_is_at_least_1280 |
| TUN | A TunSource::Create device passes check_platform |
the platform check’s error text | none |
Details that decide edge cases:
-
The idle timeout is compared as whole seconds (
Duration::as_secs).Nonemeans UDP relay is off and needs no bound;a_hysteria2_udp_idle_timeout_is_between_2_and_600_secondsaccepts 2 and 600, refuses 1 and 601, and acceptsNone. -
Shared password or user set. Were both set, a credential could have two answers.
Hysteria2InboundSpec::user_authis ignored when a shared password is set. -
Masquerade.
protocols/src/hysteria/server/masquerade.rs→Masquerade::newrefuses two things, and validation wraps its text as the reason. Answering an unauthenticated request with the success status would tell a prober that it had authenticated, so233is refused.http::StatusCode::from_u16accepts 100 to 999, and anything outside that is refused:inbound hy2: hysteria2: 233 is the authentication success status and cannot be used for the masqueradeinbound hy2: hysteria2: 1000 is not an HTTP status codehy2_handlerinbuild/inbound.rscallsMasquerade::newagain when it builds the handler; validation has already refused a bad one. WhenmasqueradeisNone, the handler usesMasquerade::default(): status 404, body404 page not found\n, content typetext/plain; charset=utf-8, the answer of an unconfigured Go server. -
TUN platform check.
protocols/src/tun/device.rs→check_platformrefuses, on every platform but Linux, a device spec withroutes, withErrorKind::Unsupported:inbound <tag>: tun routes are installed only on Linux; add them with the OS route tool. On Linux it accepts every spec. The desktoptun::open(compiled on every target but Android and iOS) calls the same function, so--testand a real start agree. On Android and iOS,tun::openrefuses every device withUnsupportedandtun devices are created by the system VPN API here; adopt its descriptor; that happens at bind time, so it is aBinderror thatcheckcannot see. A supplied device (TunSource::Fd) skips the platform check: its platform set it up. Its MTU is still checked. -
Shadowsocks 2022 server key. The server
pskmust be exactly the method’s length; the spec type says it is “decoded and sized tomethod”, so sizing is the front end’s job. etemenanki-app sizes the server key and every client key itself withnormalise_pskwhile lowering, so neitherthe shadowsocks-2022 key must be <n> bytes for this methodnorshadowsocks-2022 keys must be <n> bytes for this methodcomes from the app (see etemenanki-app). User keys are more lenient (see below).
Admission
Section titled “Admission”validate_admission checks the users an inbound would admit from its set. validate calls it for each inbound that names a set, and Actor::edit_users calls it for each inbound admitting a set being edited, so a user edit is held to the same rules as a full apply.
An inbound admits users by exactly one credential kind, build/users.rs → credential_kind:
| Inbound protocol | Credential kind |
|---|---|
| SOCKS, HTTP | Account |
| Hysteria 2 | Account with Hysteria2UserAuth::Account, Password with Hysteria2UserAuth::Password |
| Trojan, Shadowsocks | Password |
| VLESS, VMess | Uuid |
| Shadowsocks 2022 | Ss2022Psk |
| TUN | none: admission returns Ok(()) at once |
A user without that credential is skipped, here and when the inbound is built. For every other user, in key order (the set is a BTreeMap keyed by the front end’s UserId), the function derives an identity (the first table below), inserts it into seen, and runs four checks in the order of the second table, stopping at the first failure:
| Credential | Identity compared |
|---|---|
Uuid |
The 16 bytes of the UUID |
Password |
The password’s UTF-8 bytes |
Ss2022Psk |
The decoded PSK bytes as given, before normalising |
Account |
The username’s bytes, ASCII-lowercased on a Hysteria 2 inbound. The account’s password is not part of it |
| Check | Reason text | Test |
|---|---|---|
| No two users present the same identity | users <a> and <b> present the same username for accounts, users <a> and <b> present the same credential otherwise |
two_users_may_not_present_the_same_credential, hysteria2_usernames_collide_case_insensitively |
| Shadowsocks 2022: the user PSK normalises to the method | user <name>: shadowsocks-2022: PSK too short (<len> < <n>) |
a_shadowsocks_2022_user_key_must_normalise_to_the_method |
Hysteria 2 account: a name without :, and a non-empty name and password |
user <name>: a hysteria2 account needs a name without ':' and a password |
a_hysteria2_username_may_not_hold_a_colon |
| Hysteria 2 by password: the password is not empty | user <name>: a hysteria2 password must not be empty |
a_hysteria2_password_may_not_be_empty |
Each is Invalid { resource: Inbound(tag) }, printed inbound <tag>: <reason>. Why:
- One credential, one owner. If two users presented the same credential, whichever the table matched would decide who the traffic is attributed to.
<a>is the user seen first in key order and<b>the second. - Only the admitting kind counts. Two users who share a password are no conflict on a SOCKS inbound, which admits by account (
a_shared_credential_of_another_kind_is_no_conflict). - Hysteria 2 folds case. Hysteria 2 compares usernames case-insensitively on the wire (
Authenticator::user_passlowercases withto_ascii_lowercasetoo), soAliceandaliceare one username there and only there. On a SOCKS inbound the same two names are distinct. - The Hysteria 2 wire form is
user:pass. A name holding a colon could never be presented, and an empty part is no credential. SOCKS carries the name in its own field and accepts a colon (a_hysteria2_username_may_not_hold_a_colonchecks a SOCKS inbound). - A longer Shadowsocks 2022 user key is folded, not refused:
normalise_psktakes the firstkey_lenbytes of its SHA-256. Each inbound admitting the user normalises the key to its own method’s length, which is why the check lives in admission and not on the user.
Admission texts never print a password, UUID or key, and key errors print lengths only. A user is named by UserId::name(), a supervisor/src/entity/user.rs → UserName (Email or Username), whose Display is the bare string:
| Front end | UserId |
name() |
|---|---|---|
| etemenanki-app | app/src/lower.rs → UserKey, which is UserName itself |
The user’s email. A SOCKS or HTTP account, and a Hysteria 2 user without an email, is named by the username it authenticates as, so for those users the text shows that username |
| katana | src/lower/mod.rs → Uid(i64), the panel user id |
UserName::Username of the id in decimal |
Captured from the app:
configuration invalid: inbound t: users a@example.com and b@example.com present the same credentialconfiguration invalid: inbound ss: users alice and bob present the same credentialconfiguration invalid: inbound ss: user alice: shadowsocks-2022: PSK too short (15 < 16)configuration invalid: inbound hy2: users Alice and alice present the same usernameconfiguration invalid: inbound hy2: user a:b: a hysteria2 account needs a name without ':' and a passwordconfiguration invalid: inbound hy2: user a: a hysteria2 account needs a name without ':' and a passwordThe last line is a user with an empty password.
Errors after validation
Section titled “Errors after validation”Disruptive
Section titled “Disruptive”Actor::apply checks the plan right after plan returns and before it prepares anything:
let plan = plan(&self.state, &spec)?;if !options.allow_disruptive && let Some((inbound, reason)) = plan.disruptions().next(){ return Err(ApplyError::Disruptive { inbound: inbound.clone(), reason: reason.clone() });}Only the first disruption in step order is reported. Plan::disruptions() yields the Step::Disrupt steps in the order of the plan, and plan collects them in a list of handler swaps that it appends after Step::PublishPlane, one inbound at a time in spec order. The reported disruption is therefore the first disruptive inbound in spec order. The text is inbound <tag>: <reason>; this ends its live connections and needs allow_disruptive, for example inbound hy2-in: the hysteria2 obfuscation changed; connected clients cannot follow the new key; this ends its live connections and needs allow_disruptive. Which changes are disruptive, and every reason text, are on Planning and applying a change. A refused disruptive spec binds and builds nothing. Tests: a_hysteria2_obfuscation_change_needs_allow_disruptive and a_tun_device_change_needs_allow_disruptive in supervisor/tests/hot_swap.rs.
The spec was valid, but constructing one of its resources failed. Actor::prepare wraps each failure with the resource it was building:
| Resource in the text | Built by | Typical cause |
|---|---|---|
dns |
build/dns.rs → build_dns |
A resolver that does not build, or an unreadable system hosts file for a split spec with use_hosts |
outbound <tag>@v<n> |
build/outbound.rs → build_outbound |
A CA bundle that does not parse |
balancer <tag> |
Balancer::new |
An empty member list, which validation refuses first |
route |
build/route.rs → compile_routes |
Geodata missing, unreadable or without a referenced code |
inbound <tag> |
build_handler, prepare_swap, Pending::pair, user_table, prepare_users |
A certificate or key that does not parse; a QUIC endpoint that cannot open on its socket; a TUN device that cannot be prepared |
inbound <tag> |
user_table, through protocols/src/ss_2022/users.rs → Validator::from_config |
A Shadowsocks 2022 inbound on 2022-blake3-chacha20-poly1305 that admits at least one user: shadowsocks-2022: multi-user requires an aes-gcm method (InvalidInput). This is the one semantic rule that validate does not check |
Captured from the app:
configuration invalid: building inbound v failed: no certificate in PEM bundleconfiguration invalid: building inbound hy2 failed: hysteria2: the certificate file contains no certificatesconfiguration invalid: building outbound t-out@v1 failed: no certificate in CA PEM bundleconfiguration invalid: building route failed: a geosite matcher is used but no geosite file is configuredconfiguration invalid: building inbound ss failed: shadowsocks-2022: multi-user requires an aes-gcm methodSome builders repeat a rule validation already enforces, as a backstop. Each fails as Build if it is ever reached:
| Backstop | Text | Repeats |
|---|---|---|
build/inbound.rs → hy2_handler |
Masquerade::new’s text |
The masquerade rule |
build/inbound.rs → user_table |
inbound <tag>: user <label>: <e> around normalise_psk’s error |
The Shadowsocks 2022 user key rule |
build/inbound.rs → mismatch |
inbound <tag>: protocol and bind do not match |
The protocol and bind pairing: a TUN protocol whose bind is not a device, or a stream protocol whose user table came out as a Hysteria 2 table or none |
build/inbound.rs → build_transport |
missing tls |
The inbound TLS pairing |
build/outbound.rs → transport |
missing tls, missing ws host, missing grpc authority |
The outbound TLS pairing and the WebSocket and gRPC rules |
system/listener.rs → check_obfs |
malformed: hysteria2 obfs psk is too short, from Salamander::new, as InvalidInput |
The Salamander key length |
system/listener.rs → mismatch |
the handler is not of the kind its listener serves |
A handler or user table of another kind than its listener, in Pending::pair, prepare_swap and prepare_users |
topology/balancer.rs → Balancer::new |
a balancer needs at least one outbound |
The empty balancer rule |
check_obfs runs in prepare (in Pending::pair and prepare_swap) so that switching a live listener to a new key cannot fail at commit.
Validator::from_config has one more backstop, which does not fail: when two admitted users’ keys are equal once normalised, it keeps the first, skips the second and logs shadowsocks-2022: users "<a>" and "<b>" share a key; only "<a>" is matched at WARN. The protocols test users_sharing_a_psk_resolve_to_the_first pins it.
system/listener.rs → bind failed for a new listener: a TCP, UDP or Unix bind, a Unix path that holds something other than a socket, creating a TUN device (tun::open), adopting a supplied one (tun::adopt), or duplicating the device’s descriptor (try_clone). The text is inbound <tag>: binding <bind> failed: <io error>. Only listener::bind produces Bind. What Pending::pair does with a bound socket (open the Hysteria 2 QUIC endpoint, prepare a TUN device) fails as Build.
At a Unix path, bind looks at what is there first (symlink_metadata):
| At the path | bind |
|---|---|
| Nothing | Binds |
| A socket | Treats it as stale, left by a crashed run: removes it and binds. A live listener of the same supervisor on that path has the same bind, so the plan keeps it and never binds it again |
| Anything else | Refuses with ErrorKind::AlreadyExists and <path> exists and is not a socket, for example inbound socks: binding unix:/run/etemenanki/socks.sock failed: /run/etemenanki/socks.sock exists and is not a socket |
bind logs at INFO when it takes a TUN device: inbound <tag> owns tun device <name> for a created one, inbound <tag> serves a supplied tun device for an adopted one. Those lines come from prepare, so they appear even for an apply that is refused afterwards. inbound <tag> listening on <bind> is logged only when commit starts the listener.
Stopped
Section titled “Stopped”Every Supervisor method that changes or reads what the actor owns (apply, apply_with, update, update_with, set_users, upsert_user, remove_user, close, sessions, shutdown) goes through ask, which sends a Command on the actor’s channel (capacity 16) and awaits a oneshot reply. Either step failing becomes Stopped: the actor has shut down, or it dropped the reply. Command::Update and edit_users also return Stopped when there is no running spec to edit. close and sessions turn Stopped into 0 and an empty list, and shutdown ignores it.
The user path
Section titled “The user path”set_users, upsert_user and remove_user do not run validate: they change one user set and nothing else. Actor::edit_users fails with:
| Case | Error | Text |
|---|---|---|
| No running spec | Stopped |
the supervisor has shut down |
| No set carries the tag | Invalid { resource: UserSet(tag) } |
user set <tag>: no such user set |
| An inbound admitting the set refuses a user | Invalid { resource: Inbound(tag) } |
the admission texts |
| A user table does not build or does not fit its listener | Build { resource: Inbound(tag) } |
building inbound <tag> failed: <io error> |
remove_user for a user who is not in the set returns Ok(false) and checks nothing. Every table is prepared before any is stored, so a refused edit stores none. Users, principals and sessions covers the rest of that path.
The dry run: check
Section titled “The dry run: check”check runs the first half of an apply on a throwaway actor:
pub async fn check<U: UserId>(spec: &Spec<U>) -> Result<(), ApplyError> { let mut actor = Actor::new(SocketOptions::default()); let plan = plan(&actor.state, spec)?; actor.prepare(&plan, spec, false).await.map(drop)}Actor::new builds the actor’s state (root token, TaskTracker, Sessions, Tracker, sampler) and spawns nothing. Its running state is RunningState::empty(), so the plan builds every resource and reuses none, and it has no Disrupt step. prepare then runs with bind = false, and whatever it built is dropped when check returns.
What check covers:
| Step | Done | Can fail with |
|---|---|---|
validate |
Every rule on this page | The validation variants |
build_dns |
The resolvers; for a split spec with use_hosts, reads the system hosts file |
Build on dns |
build_outbound for each outbound |
Transport connectors, TLS client configs and CA bundles, protocol keys | Build on outbound <tag>@v<n> |
Balancer::new for each balancer |
Members with their probe destinations; no probe task is started | Build on balancer <tag> |
compile_routes |
The route table, and the geodata sets the rules reference, read from their files | Build on route |
admit for each inbound |
Admissions and user keys | none |
build_handler for each inbound |
Server TLS configs from the PEM bytes, the Hysteria 2 QUIC server config and masquerade, user tables (including Validator::from_config) |
Build on inbound <tag> |
What check cannot see:
| Not done | Where it happens on a start | Fails with |
|---|---|---|
| Binding TCP, UDP and Unix listeners, including the check that a Unix path holds no other file | listener::bind |
Bind |
Creating a TUN device and installing its routes (tun::open), or adopting a supplied descriptor (tun::adopt) |
listener::bind |
Bind |
Opening the Hysteria 2 endpoint on its socket (Hy2Inbound::open), checking its obfuscation key (check_obfs), preparing a TUN device (TunInbound::prepare) |
Pending::pair |
Build |
| A disruption against a running supervisor | Actor::apply |
Disruptive |
| Anything on the network: resolving upstreams, reaching them, probing balancer members | after commit | not an ApplyError |
So check and a start agree on every semantic rule and on every constructor except the bind-time ones. A spec that passes check can still be refused by a running supervisor as Disruptive, because check plans from nothing. The app demonstrates the bind gap: a SOCKS inbound whose listen names an existing regular file passes --test with Configuration OK., and a start refuses it as Bind.
check runs on the caller’s task. It takes no SupervisorBuilder settings: its actor uses SocketOptions::default(). The socket policy has no effect there, because check dials nothing.
Callers:
| Caller | Function | What it checks |
|---|---|---|
etemenanki-app --test |
app/src/instance.rs → check → check_bytes |
The config file and the subscribe file it names, merged and lowered, and the [api] options (crate::api::options), then etemenanki_supervisor::check on the spec. A bad [api] therefore fails --test as LoadError::Config |
| etemenanki-app REST API | app/src/api.rs → ConfigFile::set_route → check_bytes |
The edited config, before it is written |
| katana | src/runtime.rs → check_node → lower::test_spec |
The process-wide outbounds and DNS and the node’s own route table (without audit rules), plus the locally configured inbound of a Hysteria 2 node |
| Tests | app/tests/unit/subscribe.rs → every_lowered_node_builds_and_the_dns_setup_with_it; katana tests/unit/lower/outbound.rs → wireguard_ipv4_only_builds_with_ipv4_address, wireguard_ipv6_only_requires_ipv6_address |
A lowered subscribe example; katana’s WireGuard lowering |
In katana, build_node (used at startup and for the nodes a reload adds), test_config (--test) and apply_reload call check_node, each after building the node’s panel client. For a Hysteria 2 node, test_spec lowers the inbound from [node.hysteria] (local_hysteria_node, with a port of 0 replaced by 1) and admits one placeholder user (uid 0, email placeholder). For every other node type, the inbound and its users come from the panel and are not part of the check.
How front ends report the errors
Section titled “How front ends report the errors”The supervisor returns an ApplyError and logs nothing about a refusal. Its only log line about an apply is the DEBUG line applied: <n> built, <n> reused, <n> swapped, <n> drained at the end of a commit (supervisor.rs → Actor::commit). Each front end adds its own context.
etemenanki-app
Section titled “etemenanki-app”app/src/instance.rs → LoadError joins the two sources of failure, both #[error(transparent)], so the text is the inner error’s:
pub enum LoadError { /// The files could not be read, or what they say could not be lowered. Config(#[from] io::Error), /// The supervisor refused the spec. Apply(#[from] ApplyError),}The app’s lowering (app/src/lower.rs → lower) refuses syntax and file problems itself (LoadError::Config) and leaves the rules on this page to the supervisor, with two exceptions:
- Shadowsocks 2022 server and client keys. The spec wants them sized to the method, so the lowering sizes them with
normalise_psk: it folds a longer key and refuses a short one, asinbound <tag>: shadowsocks-2022: PSK too short (<len> < <n>)oroutbound <tag>: shadowsocks-2022: PSK too short (<len> < <n>). A user’s key is decoded but not sized, because its size depends on the inbound admitting it, so the supervisor’s admission check sizes it. - User names. A user set is a map, so a second entry under a taken name would replace the first. The lowering refuses a user without the name its protocol keys it by (
inbound <tag>: users[<i>] has no email; a user of this inbound is named by its email, or… has no userfor an account) and two entries under one name (inbound <tag>: users[<i>] and users[<j>] are both named "<name>";accounts[…]for SOCKS and HTTP). Two names presenting one credential are left to admission.
app/tests/unit/lower.rs → semantic_problems_are_left_to_the_supervisor lowers a config that breaks many of this page’s rules (a duplicate inbound tag, a TUN MTU of 1000, a Hysteria 2 inbound with both a password and users, a 1-second UDP idle timeout and zero circuits, a shared Trojan password, a WireGuard family without a matching address, a Hysteria 2 client with zero streams, a balancer and a rule naming a missing outbound), and asserts that lowering accepts it and carries those values into the spec. syntax_errors_are_refused pins what the app refuses on its own. The lowering itself is on etemenanki-app: from TOML to a spec.
Captured from the app, the lowering’s own refusals print without the supervisor’s wording:
configuration invalid: inbound ss: shadowsocks-2022: PSK too short (16 < 32)configuration invalid: outbound ss-out: shadowsocks-2022: PSK too short (15 < 16)configuration invalid: inbound t: users[0] and users[1] are both named "alice@example.com"configuration invalid: inbound t: users[1] has no email; a user of this inbound is named by its emailThe app’s tracing_subscriber::fmt subscriber writes to stdout, ERROR lines included. The --test and start lines are logged by target etemenanki_app (app/src/main.rs); config loaded, config reloaded and the reload errors by target etemenanki_app::instance.
| Where | Output |
|---|---|
--test, accepted |
Configuration OK. on stdout, exit status 0 |
--test, refused |
configuration invalid: <e> at ERROR, exit status 1 |
| Start, refused | failed to start: <e> at ERROR, exit status 1 |
| Start, API cannot start | failed to start: API on <listen>: <e> at ERROR, exit status 1 |
| Start, applied | config loaded: <summary> at INFO |
| Reload, applied | config reloaded: <summary> at INFO |
| Reload, files unreadable | reload: cannot read <path>: <e> at ERROR |
Reload, refused as Disruptive |
reload refused, keeping the running config: inbound <tag>: <reason>, which would end its live connections; restart to apply it at ERROR |
| Reload, any other refusal | reload: <e>; keeping the running config at ERROR |
The app starts with SupervisorBuilder::start (Instance::start → Core::start with Supervisor::builder()) and reloads with Supervisor::apply, both with the default ApplyOptions, so allow_disruptive is never set: it refuses a reload that changes a TUN inbound (its device or its settings) or a Hysteria 2 listener’s obfuscation, and a restart applies it. <summary> is the summary format described under ApplyReport. The reload path is on etemenanki-app: running, reloading and shutting down.
For the REST API, impl From<LoadError> for ControlError sorts failures by fault. Bind and Stopped become ControlError::Failed (HTTP 500): not the config’s fault. Every other ApplyError becomes ControlError::Invalid (HTTP 422). An io::Error from lowering is Invalid when its kind is InvalidData or InvalidInput, and Failed otherwise. The other two variants do not come from a load: UnknownGroup (HTTP 404, no route group named "<group>") and Stale (HTTP 409, the config file has changed since it was read; its ETag is now <etag>). webclient/src/router.rs maps each variant to its status. ConfigFile::set_route refuses as Stale in two places: before the edit, when If-Match names another ETag, and after check_bytes, when the file on disk no longer holds the bytes it read. It re-reads the file at that point because the check builds everything a start would and takes a while, and a hand edit saved meanwhile must not be overwritten.
The FFI
Section titled “The FFI”ffi/src/error.rs → impl From<LoadError> for FfiError first picks out the client’s own refusal of a server inbound, FfiError::ServerInbound, printed inbound "<tag>" serves <protocol>, a server protocol; only a tun inbound and local socks or http inbounds run in a client. Everything else goes through ControlError:
ControlError |
FfiError |
|---|---|
Invalid |
Config { message } |
Stale |
Config, with the Stale text as its message |
UnknownGroup |
UnknownGroup { group } |
Failed |
Io { message } |
Config and Io print the message alone, so a mobile app sees the ApplyError text unchanged. Because ApplyError::Stopped becomes ControlError::Failed, it reaches the app as FfiError::Io with the text the supervisor has shut down, not as FfiError::Stopped (Mobile library (etemenanki-ffi)).
katana
Section titled “katana”--test(src/runtime.rs→test_config) first refuses a config without nodes (config defines no [[node]] entries), runswarn_shared_wireguard(aWARNwhen several nodes route to one WireGuard outbound), then builds each node’s panel client and runscheck_node, prefixing any error with the node id. It printsConfiguration OKon stdout, orconfiguration error: <e>on stderr with exit status 1. Captured:configuration error: node 1: outbound wg-egress@v0: wireguard address_family ipv6_only needs an IPv6 address.- A config file reload (
apply_reload) runscheck_nodefor each new or changed node, and for every node when[[outbound]]or[dns]changed, before it touches any running one. A refusal logsreload: node <node>: <e>; keeping current configatERROR, where<node>is<panel_type>@<host>#<node_id>, followed by/<node_type>when one is set. See Process runtime and reload. - A node applies its spec with
Supervisor::apply_withandApplyOptions { allow_disruptive: true }(src/manager/node.rs→reconcile), so katana never receivesDisruptive, and a changed Hysteria 2 obfuscation is applied rather than refused. See Node manager. - katana’s lowering of a Hysteria 2 inbound (
src/lower/inbound.rs→lower_hysteria) keeps three checks of its own that the supervisor also makes: the masquerade, throughMasquerade::new(its comment: the supervisor refuses the same responses, and katana judges them too “so the refusal reads as it always has”), the Salamander key length (obfs_password must be at least 4 bytes for salamander), and the 2 to 600 second UDP idle timeout (upstream’s range). These texts carry noinboundprefix, for exampleconfiguration error: node 1: udp_idle_timeout must be between 2 and 600 secondsandconfiguration error: node 1: obfs_password must be at least 4 bytes for salamander. The supervisor’s rules still apply behind them. The same function refuses settings the supervisor has no rule for:udp_idle_timeout is set but udp is not enabled,obfs_password is set but obfs is not; did you mean obfs = "salamander"?,unknown obfs "<x>" (expected "salamander")andunknown hysteria credential kind "<x>" (expected "uuid" or "user_pass"). Lowering inbounds and outbounds covers that file. - katana’s user lowering (
src/lower/mod.rs→lower_users) skips, with aWARN, a user whose credential cannot be lowered (skipping user <uid>: <why>), a uid listed twice (skipping user <uid>: listed twice), and a user whose credential an earlier user already presented (skipping user <uid>: shares its credential with user <first>). The first user keeps the credential, so two panel users who share one do not reachvalidate_admission. The reasons name the user and never the credential.
Invariants
Section titled “Invariants”| Invariant | Why | Pinned by |
|---|---|---|
| Every apply, update, start and check validates before it plans a step | plan calls validate first, and every path calls plan |
supervisor/tests/unit/plan.rs → an_invalid_spec_is_refused_without_a_plan |
| A refused spec changes nothing that runs | Validation and Disruptive refuse before prepare; prepare has no visible effect and its results are dropped |
supervisor/tests/hot_swap.rs → a_refused_spec_changes_nothing: an unknown route target, and a spec that also swaps the user set and the route default but cannot bind a port, both leave the epoch, the users and the listeners as they were |
| Validation touches no file, socket or clock | Pure functions over the spec | The unit tests build every spec in memory with arbitrary TLS bytes |
| Each rule is tested alone | Each test breaks one rule in an otherwise valid spec and, where the rule has an edge, checks the edge that is still accepted | supervisor/tests/unit/validate.rs (module comment) |
| The builders can assume validity | The expect calls in Actor::prepare rely on validation |
the tag and balancer tests |
| Outbound and balancer tags share one namespace; inbound and user set tags do not | A route names outbounds and balancers alike | a_balancer_may_not_share_an_outbounds_tag, an_inbound_may_share_an_outbounds_tag |
| No spec tag is empty | The empty tag names the supervisor’s own targets | no_tag_may_be_empty |
| A user edit meets the same admission rules as an apply | edit_users calls validate_admission |
no dedicated test |
| Validation texts never print a password, UUID or key | Users are named by UserId::name(); key errors print lengths only. A name may be an account’s username (etemenanki-app) |
no dedicated test |
check and a start agree on semantics and construction |
The same plan and prepare, differing only in bind |
no dedicated test |
Failure paths and cancellation
Section titled “Failure paths and cancellation”- One error per refusal.
validatereturns at the first broken rule, andprepareat the first failed step. A spec with several problems needs as many attempts to find them. - What a refusal releases. Everything
preparebuilt lives in local values and thePreparedstruct. On an error they are dropped: listeners bound earlier in the same prepare close again (a_refused_spec_changes_nothingbinds a spare port first and checks that it is free afterwards), and a Unix socket file created by such a bind is removed by itsSocketFileguard. User keys are handed out on a staged copy ofUserKeys, kept only if the apply commits. - Dropping an apply future.
askfirst sends the command, then awaits the reply. A caller that drops the future while the send is still waiting for channel capacity sends nothing. Once the command is in the channel, the actor runs the apply to the end and commits it if it succeeds; the reply is then sent to nobody and ignored (let _ = reply.send(...)). checkis an ordinary future on the caller’s task. It spawns nothing, so dropping it drops everything it built.- The first spec.
SupervisorBuilder::startapplies the first spec before it spawns the sampler, the usage sink or the actor task. A refused first spec starts nothing and returns theApplyError. Before it applies anything,startasserts that the sample interval is not zero; a zero interval panics withthe sample interval must not be zeroinstead of returning anApplyError. - Logs from a refused apply. The supervisor logs no refusal, but prepare’s own
INFOlines stay in the log: a TUN device created or adopted while preparing an apply that is refused afterwards has already loggedinbound <tag> owns tun device <name>orinbound <tag> serves a supplied tun device.
Limits
Section titled “Limits”| Limit | Value | Where |
|---|---|---|
| Errors reported per refusal | 1 | validate, prepare |
| TUN MTU | at least 1280 (MIN_TUN_MTU) |
validate_inbound |
| Hysteria 2 UDP idle timeout | 2 to 600 whole seconds, inclusive (HY2_UDP_IDLE), when set |
validate_inbound |
Hysteria 2 max_connections, max_circuits |
at least 1 | validate_inbound |
Hysteria 2 client max_concurrent_streams |
at least 1 | validate_outbound |
| Salamander key | at least 4 bytes (MIN_SALAMANDER_PSK) |
obfs |
| Masquerade status | 100 to 999, except 233 | Masquerade::new |
| Shadowsocks 2022 server key and client keys | exactly 16 bytes (2022-blake3-aes-128-gcm) or 32 bytes (2022-blake3-aes-256-gcm, 2022-blake3-chacha20-poly1305) |
validate_inbound, validate_outbound |
| Shadowsocks 2022 user key | at least the method’s length; longer keys are folded | validate_admission |
| Shadowsocks 2022 inbound admitting users | 2022-blake3-aes-128-gcm or 2022-blake3-aes-256-gcm only |
Validator::from_config, at build time |
| Supervisor command channel | 16 queued commands | SupervisorBuilder::start |
supervisor/tests/unit/validate.rs is compiled into the crate (#[path] from build/validate.rs), so cargo test -p etemenanki-supervisor --lib runs it. It builds a valid base spec (one SOCKS inbound on 127.0.0.1:1080, one freedom outbound direct, the system resolver) and breaks one rule at a time.
| Test | Pins |
|---|---|
the_base_spec_is_valid |
The base spec passes |
tags_are_unique_within_each_kind |
DuplicateTag for inbounds, outbounds, balancers and user sets |
a_balancer_may_not_share_an_outbounds_tag |
The shared namespace |
an_inbound_may_share_an_outbounds_tag |
Other kinds live apart |
no_tag_may_be_empty |
Invalid on the inbound, the outbound and the user set |
a_route_rule_must_name_a_known_outbound_or_balancer |
UnknownReference from Route |
the_route_default_must_name_a_known_outbound_or_balancer |
The same for the default |
a_route_may_name_a_balancer |
A balancer is routable |
a_balancer_member_must_be_a_known_outbound |
UnknownReference from the balancer |
a_balancer_member_may_not_be_another_balancer |
Members are outbounds only |
an_inbound_must_name_a_known_user_set |
UnknownReference from the inbound, kind UserSet |
a_balancer_needs_a_member |
EmptyBalancer |
a_balancer_member_needs_an_upstream_a_tcp_probe_can_reach |
UnprobeableMember for freedom, blackhole, Hysteria 2 and WireGuard |
two_inbounds_may_not_share_a_bind |
The later inbound is invalid |
the_same_port_over_udp_is_another_bind |
TCP and UDP binds differ |
each_protocol_is_served_only_on_its_kind_of_bind |
Six refused pairings, three accepted |
an_inbound_carries_tls_material_exactly_when_its_shape_layers_tls |
Eight shape and material combinations |
an_inbound_needs_no_ws_host_or_grpc_authority |
Inbound gaps are accepted |
an_outbound_carries_tls_settings_exactly_when_its_shape_layers_tls |
Six combinations |
an_outbound_needs_a_ws_host_and_a_grpc_authority |
Outbound gaps are refused |
a_unix_listener_carries_only_the_plain_tcp_shape |
TLS and WebSocket shapes refused on a Unix bind |
trojan_vless_and_vmess_need_a_user_set |
No open mode; naming a set satisfies the rule |
hysteria2_takes_a_shared_password_or_a_user_set_but_not_both |
Both and neither refused |
a_hysteria2_shared_password_may_not_be_empty |
Empty shared password |
a_hysteria2_udp_idle_timeout_is_between_2_and_600_seconds |
1 and 601 refused, 2, 600 and None accepted |
hysteria2_serves_at_least_one_connection_and_one_circuit |
Zero refused, one accepted |
a_salamander_key_is_at_least_4_bytes_on_either_side |
3 bytes refused, 4 accepted, inbound and outbound |
a_masquerade_may_not_answer_with_the_success_status |
233 refused, 404 accepted |
a_tun_device_mtu_is_at_least_1280 |
1279 refused, 1280 accepted |
socks_over_a_unix_socket_needs_a_udp_bind_to_serve_udp |
The Unix and UDP rule and its three accepted edges |
a_shadowsocks_2022_server_key_is_sized_to_its_method |
One byte short or long refused |
a_wireguard_address_family_needs_an_address_of_that_family |
Five address and family combinations |
a_hysteria2_client_needs_a_password_and_a_stream |
Empty password, zero streams |
shadowsocks_2022_client_keys_are_sized_to_their_method |
Both the user key and an identity key |
two_users_may_not_present_the_same_credential |
A shared Trojan password |
a_shared_credential_of_another_kind_is_no_conflict |
Only the admitting kind counts |
hysteria2_usernames_collide_case_insensitively |
Alice and alice on Hysteria 2 only |
a_hysteria2_username_may_not_hold_a_colon |
Hysteria 2 only |
a_hysteria2_password_may_not_be_empty |
By-password admission; a UUID-shaped password is accepted |
a_shadowsocks_2022_user_key_must_normalise_to_the_method |
A short key refused, a longer one folded and accepted |
Other tests that pin this page:
| File | Test | Pins |
|---|---|---|
supervisor/tests/unit/plan.rs |
an_invalid_spec_is_refused_without_a_plan |
plan refuses before planning |
supervisor/tests/hot_swap.rs |
a_refused_spec_changes_nothing |
UnknownReference and Bind refusals leave everything running as it was. The unbindable spec also replaces the user set (alice with bob) and routes to block; afterwards alice still connects and is routed direct, bob is refused, the epoch is unchanged, and a spare port bound earlier in the same prepare is free again |
supervisor/tests/hot_swap.rs |
a_hysteria2_obfuscation_change_needs_allow_disruptive |
Disruptive for an obfuscation change |
supervisor/tests/hot_swap.rs |
a_tun_device_change_needs_allow_disruptive |
Disruptive for a TUN change (needs CAP_NET_ADMIN; skipped otherwise) |
protocols/tests/unit/hysteria/server/masquerade.rs |
the_authentication_success_status_is_refused, a_value_that_is_not_a_status_code_is_refused |
The two masquerade refusals; 0, 99, 1000 and 65535 refused, 599 accepted |
protocols/tests/unit/ss_2022/users.rs |
users_sharing_a_psk_resolve_to_the_first |
The Validator::from_config backstop keeps the first of two users with one key |
app/tests/unit/lower.rs |
semantic_problems_are_left_to_the_supervisor, syntax_errors_are_refused |
The split between the app and the supervisor |
app/tests/unit/subscribe.rs |
every_lowered_node_builds_and_the_dns_setup_with_it |
A lowered subscribe example passes check |
katana tests/unit/lower/outbound.rs |
wireguard_ipv4_only_builds_with_ipv4_address, wireguard_ipv6_only_requires_ipv6_address |
katana’s WireGuard lowering against check |
Rules with no dedicated test: an empty balancer tag, an empty Hysteria 2 account name or password (only the colon is tested), the TUN platform check, a Shadowsocks 2022 inbound admitting users on 2022-blake3-chacha20-poly1305, the user path’s no such user set, and a user edit refused by admission (the admission tests call validate and validate_admission directly, and the only set_users call in the suites succeeds). No supervisor test matches an error’s text; only the protocols crate’s masquerade test and katana’s WireGuard test match a fragment of one. Rewording a rule therefore passes the suites; update this page and Error messages with it. How to run the suites is on Testing.