Route model
Source files: 28 · checked against Etemenanki 596916d · katana v3.0.1
Etemenanki/environment/src/lib.rsEtemenanki/environment/src/routing.rsEtemenanki/environment/tests/unit/routing.rsEtemenanki/app/src/router.rsEtemenanki/app/src/config.rsEtemenanki/app/src/instance.rsEtemenanki/app/src/connector.rsEtemenanki/app/src/serve.rsEtemenanki/app/src/inbound/tun.rsEtemenanki/app/src/flow.rsEtemenanki/app/src/outbound/mod.rsEtemenanki/app/src/outbound/udp_fanout.rsEtemenanki/protocols/src/flow.rsEtemenanki/protocols/src/sniff/mod.rsEtemenanki/concepts/src/sniff.rsEtemenanki/app/tests/unit/router.rsEtemenanki/app/tests/unit/config.rsEtemenanki/app/tests/integration/e2e_route_context.rsEtemenanki/app/tests/integration/e2e_sniff.rsEtemenanki/app/tests/integration/e2e_udp_route.rskatana/src/router.rskatana/src/config.rskatana/src/connector.rskatana/src/runtime.rskatana/src/manager/node.rskatana/tests/unit/connector.rskatana/tests/unit/e2e.rskatana/tests/integration/sniff.rs
The route model decides which outbound a flow goes to. It lives in etemenanki-environment as the module etemenanki_environment::routing (environment/src/routing.rs) and has two consumers: the standalone app (app/src/router.rs) and katana (src/router.rs). Each consumer compiles its own rule tables ([[route.rule]] in the app, [[node.route.rule]] in katana) into the same RouteTable, and both call the same pick.
Read this page before you add a matcher, change how a domain is compared, touch the geodata loader, or change the context a consumer passes to the router. The operator’s view of the same rules is in Etemenanki routing and katana routing.
Responsibilities
Section titled “Responsibilities”The module does four things:
- Matching.
RouteTable::pickwalks an ordered list of rules and returns the output of the first rule that has at least one matching matcher, or the default output. - Geodata loading.
build_geo_datadecodes v2ray/Xraygeoip.datandgeosite.datfiles (protobuf, throughprost) and keeps only the codes the table references. - Parsing helpers.
parse_port_matchandparse_domain_regexturn config strings into validated matchers, so a consumer rejects bad input while it builds the table. - Composition.
Routerbundles a table with itsGeoData. A consumer holds oneRouterper generation.
It deliberately leaves the following to its callers:
| Not here | Where it is |
|---|---|
The config format. The module never sees TOML; each consumer maps its own rule struct to RouteMatch values. |
app app/src/router.rs → build_router, katana src/router.rs → build_router |
Resolving outbound tags. RouteTable is generic over Out and stores Arc<Out>. |
the consumers’ build_router |
| DNS. A domain target is matched by name only. It is never resolved, so a CIDR or geoip rule never sees the address a domain will dial. | the outbounds, see Dialers and socket policy |
| Sniffing. The module accepts a sniffed domain but never inspects a payload. | the protocols crate, see Sniffing |
| UDP fan-out. The module routes one target at a time; splitting an association per packet is the consumer’s job. | app app/src/outbound/udp_fanout.rs → FanOutLink, katana src/connector.rs → FanOut |
pick is a pure function of the target, the table and the geodata. The module performs no I/O apart from reading the two .dat files inside build_geo_data, spawns no task, holds no lock and has no async code. It inherits the crate-wide lint gate from environment/src/lib.rs: #![deny(clippy::unwrap_used, clippy::expect_used, clippy::indexing_slicing, clippy::arithmetic_side_effects)], relaxed only under cfg(test).
Key types
Section titled “Key types”Remote and TargetNetwork
Section titled “Remote and TargetNetwork”pub enum Remote<'a> { IpAddr(IpAddr), Domain(&'a str),}
pub enum TargetNetwork { Tcp, Udp,}Remote is a borrowed view of the destination. It is decoupled from etemenanki_concepts::net::Remote so the route model carries no opinion about a consumer’s network types; each consumer converts its own Destination in a route_target function. Remote derives Debug, Clone, Copy; TargetNetwork adds PartialEq, Eq.
RouteTarget
Section titled “RouteTarget”pub struct RouteTarget<'a> { pub remote: Remote<'a>, pub port: u16, pub network: Option<TargetNetwork>, pub source: Option<IpAddr>, pub sniffed_domain: Option<&'a str>, pub inbound_tag: Option<&'a str>,}
impl<'a> RouteTarget<'a> { pub fn new(remote: Remote<'a>, port: u16) -> Self; pub fn with_network(mut self, network: TargetNetwork) -> Self; pub fn with_source(mut self, source: IpAddr) -> Self; pub fn with_sniffed_domain(mut self, domain: &'a str) -> Self; pub fn with_inbound_tag(mut self, tag: &'a str) -> Self;}remote and port are what every request has; new sets the four Option fields to None. The with_* builders each set one of them. RouteTarget derives Debug, Clone, Copy, so it is passed by value on the hot path. The fields are public, but consumers build targets with new and the builders so that adding a field later does not break them.
| Field | Read by | When it is None |
|---|---|---|
remote |
every domain matcher (when it is a Domain), Cidr and GeoIp (when it is an IpAddr) |
never; always set |
port |
PortRange |
never; always set |
network |
Network |
a Network matcher does not match |
source |
SourceCidr |
a SourceCidr matcher does not match |
sniffed_domain |
every domain matcher, including GeoSite |
only remote is tried |
inbound_tag |
InboundTag |
an InboundTag matcher does not match |
The rule that holds the model together: a matcher whose field was not supplied does not match. The module never guesses a network, a source or a domain. The cost is that an unsupplied field turns a rule into one that silently never fires, which is why the app has end-to-end tests proving that each context field its config accepts actually reaches the router (see Tests).
RouteMatch and DomainRegex
Section titled “RouteMatch and DomainRegex”pub enum RouteMatch { GeoSite(CompactString), DomainSuffix(CompactString), DomainKeyword(CompactString), DomainFull(CompactString), DomainRegex(DomainRegex), GeoIp(CompactString), Cidr(IpCidr), SourceCidr(IpCidr), PortRange(u16, u16), Network(TargetNetwork), InboundTag(CompactString),}RouteMatch derives Debug, Clone, PartialEq, Eq. The private function matches(m, target, domains, geo) evaluates one variant. Each variant reads only the part of the target it is about; the domain matchers read both the request domain and the sniffed domain:
| Variant | Reads | Matches when |
|---|---|---|
DomainSuffix(parent) |
request and sniffed domain | either domain is parent or a subdomain of it, on a label boundary (see Suffix matching) |
DomainKeyword(needle) |
request and sniffed domain | either domain contains needle as a substring |
DomainFull(name) |
request and sniffed domain | either domain equals name |
DomainRegex(re) |
request and sniffed domain | re matches either domain (unanchored is_match) |
GeoSite(code) |
request and sniffed domain | the DomainSet loaded under code matches either domain |
Cidr(c) |
remote, only when it is an IP |
c.contains(&ip) |
GeoIp(code) |
remote, only when it is an IP |
the CidrSet loaded under code matches the IP |
SourceCidr(c) |
source |
source is set and c.contains(&source) |
PortRange(lo, hi) |
port |
port >= lo && port <= hi, inclusive at both ends |
Network(want) |
network |
network == Some(want) |
InboundTag(tag) |
inbound_tag |
inbound_tag is set and equals tag |
regex::Regex implements neither PartialEq nor Eq, so the regex variant wraps it:
#[derive(Debug, Clone)]pub struct DomainRegex(regex::Regex);
impl DomainRegex { pub fn new(pattern: &str) -> io::Result<Self>; pub fn as_str(&self) -> &str;}
impl PartialEq for DomainRegex { /* self.0.as_str() == other.0.as_str() */ }impl Eq for DomainRegex {}Equality is defined on the source pattern: two regex matchers are equal when they were spelled the same way. DomainRegex::new compiles the pattern and fails with io::ErrorKind::InvalidInput and the message invalid domain regex {pattern:?}: {e}. The inner Regex is private, so the only way to get a DomainRegex is through new, and compilation always happens while a table is built, never while matching.
RouteItem and RouteTable
Section titled “RouteItem and RouteTable”pub struct RouteItem<Out> { pub matchers: SmallVec<[RouteMatch; 3]>, pub output: Arc<Out>,}
pub struct RouteTable<Out> { pub routes: Vec<RouteItem<Out>>, pub default: Arc<Out>,}
impl<Out> RouteTable<Out> { pub fn pick(&self, target: RouteTarget<'_>, geo: &GeoData) -> Arc<Out>;}A RouteItem is one rule: an OR over its matchers. Up to three matchers sit inline in the SmallVec; a longer rule spills to the heap. Outputs are Arc<Out>, so pick returns a reference-count clone, and the same outbound can be the output of several rules and the default at once. Neither type derives anything, and no bound is placed on Out.
Router
Section titled “Router”pub struct Router<Outbound> { table: RouteTable<Outbound>, geo: GeoData,}
impl<Outbound> Router<Outbound> { pub fn new(table: RouteTable<Outbound>, geo: GeoData) -> Self; pub fn route(&self, target: RouteTarget<'_>) -> Arc<Outbound>; pub fn default_outbound(&self) -> Arc<Outbound>;}route is self.table.pick(target, &self.geo). default_outbound returns a clone of the table’s default. Its doc comment still describes it as the route for datagram associations, but at the pinned revisions neither consumer calls it: both route UDP per packet through a fan-out (see Per-packet UDP routing).
Each consumer defines a local alias for it:
// etemenanki-app, app/src/router.rspub type Router = routing::Router<Outbound>;
// katana, src/router.rspub type Router<Outbound> = routing::Router<Outbound>;Parsing helpers
Section titled “Parsing helpers”pub fn parse_port_match(s: &str) -> io::Result<RouteMatch>;pub fn parse_domain_regex(pattern: &str) -> io::Result<RouteMatch>;parse_port_match accepts "80" or "1000-2000":
- It splits on the first
-and trims whitespace on each side, so" 1000 - 2000 "is accepted. - Each bound must parse as a
u16."70000"and"nope"fail withinvalid port spec: "70000"(the input is quoted with{:?}). - A lower bound above the upper bound fails with
invalid port spec: "2000-1000" has a lower bound above its upper bound. An inverted range would otherwise compile into a rule that can never match, which is indistinguishable from a rule that is never hit. - A single port
pbecomesPortRange(p, p), and"80-80"is accepted as the same thing.
All errors are io::ErrorKind::InvalidInput. parse_domain_regex is DomainRegex::new(pattern).map(RouteMatch::DomainRegex).
How pick decides
Section titled “How pick decides”flowchart TD
start(["pick(target, geo)"]) --> fold["Domains::of(target): lowercase the request and sniffed domains once"]
fold --> next{"another rule in routes?"}
next -- "no" --> def(["return default.clone()"])
next -- "yes, rule i" --> any{"any matcher of rule i matches?"}
any -- "yes, stop at the first" --> out(["return routes[i].output.clone()"])
any -- "no" --> next
Three properties follow directly from the loop in RouteTable::pick:
- The first rule wins. The loop returns on the first hit, so rule order is the only priority. A broad rule placed early shadows every later rule.
- Any matcher within a rule.
item.matchers.iter().any(...)makes a rule an OR of its matchers, and the model has no conjunction. A rule withdomain_suffixandportmatches a flow that satisfies either one, not both. - The default is the fallback. When no rule matches,
pickreturnsself.default.clone(). It cannot fail and never returns “no route”.
A rule with an empty matchers list never matches, because any over nothing is false. Both consumers accept such a rule (a [[route.rule]] with only outbound = "block" passes --test), and it is inert. A catch-all belongs in default.
Domain normalisation
Section titled “Domain normalisation”pick lowercases the target’s domains once per call, through a private helper:
struct Domains { request: Option<String>, sniffed: Option<String>,}Domains::of sets request to d.to_ascii_lowercase() when remote is Remote::Domain(d) and to None for an IP, and sets sniffed to the lowercased sniffed_domain. Every domain matcher then goes through Domains::any, which tries request first and sniffed second. Folding once per pick replaced a per-matcher fold that allocated a fresh String for every domain rule the walk passed.
Only the target side is folded here. The rule side must already be lowercase:
- The app lowercases
domain_suffix,domain_keywordanddomain_fullwithto_ascii_lowercasewhile it builds the table; katana lowercasesdomain_suffix, the only domain key it has. build_domain_setlowercases every geosite value before building its matcher.- A
RouteMatch::DomainSuffixbuilt by hand with an uppercase letter never matches anything.
The fold is ASCII only: to_ascii_lowercase leaves every non-ASCII byte as it is, so a name is otherwise compared in exactly the form the request or the payload carried.
Sniffed domains
Section titled “Sniffed domains”sniffed_domain carries a name recovered from the payload, a TLS SNI or an HTTP Host, when the request itself named only an IP. Every domain matcher, including GeoSite, tries it as well as the request domain. This is the only way a rule written in domains can match a flow addressed by IP. Cidr and GeoIp ignore the sniffed domain and read remote only, so an IP-addressed flow is still matched by its address.
The protocols crate fills Flow::sniffed (an Option<SniffedBehavior> whose domain is a CompactString) only when the inbound sniffs and worth_sniffing holds, that is when the destination is a bare IP; a destination that already names a domain is routed by that name. Both consumers pass s.domain.as_str() to with_sniffed_domain. How the payload is inspected is covered in Sniffing.
Suffix matching
Section titled “Suffix matching”DomainSuffix and geosite’s type-2 (Domain) entries share one allocation-free helper:
fn is_subdomain_of(d: &str, parent: &str) -> bool { match d.len().checked_sub(parent.len()) { Some(0) => d == parent, Some(n) => d.ends_with(parent) && d.as_bytes().get(n.saturating_sub(1)) == Some(&b'.'), None => false, }}When d is longer than parent, it must end with parent and the byte just before the suffix must be .: a label-boundary check. It replaced ends_with(&format!(".{parent}")), which allocated on every call. checked_sub and get keep it inside the crate’s no-panic lint gate.
d |
parent |
Result | Why |
|---|---|---|---|
example.com |
example.com |
match | same length and equal |
mail.example.com |
example.com |
match | ends with it, preceded by . |
notexample.com |
example.com |
no match | preceded by t, not a label boundary |
example.com.example.net |
example.com |
no match | does not end with it |
com |
example.com |
no match | shorter than parent |
a.example.com |
.example.com |
no match | the byte before .example.com is a |
example.com. |
example.com |
no match | the trailing dot is part of d |
The last two rows matter to config authors. A suffix written with a leading dot, or a target spelled as a fully qualified name with a trailing dot, does not match the usual spelling: the module strips dots on neither side.
IP, source, port, network and inbound tag
Section titled “IP, source, port, network and inbound tag”CidrandGeoIpmatch only whenremoteis an IP. A domain target returnsfalsebefore any set is consulted, so a negatedGeoIp("!code")does not match a domain target either. Thecidrcrate’sIpCidr::containscompares families strictly: an IPv4 range never contains an IPv6 address, including an IPv4-mapped one.SourceCidrreadssource, neverremote. A target without a source leaves the rule inert, even when its destination falls inside the range.PortRangeis inclusive at both ends.Networkcompares againstOption<TargetNetwork>andInboundTagagainstOption<&str>, so both are inert until the consumer supplies the field. Both consumers mapDialNetwork::TcpandDialNetwork::Udpand supply no network forDialNetwork::UnknownorDialNetwork::Unix.
Geodata
Section titled “Geodata”GeoData and its sets
Section titled “GeoData and its sets”#[derive(Default)]pub struct GeoData { pub domain: HashMap<CompactString, DomainSet>, pub ip: HashMap<CompactString, CidrSet>,}
pub struct DomainSet { matchers: Vec<DomainMatcher>,}
pub struct CidrSet { cidrs: Vec<IpCidr>, reverse: bool,}Both maps are keyed by the raw matcher string exactly as the rule wrote it, such as "cn", "google@ads" or "!cn". RouteMatch::GeoSite(raw) and RouteMatch::GeoIp(raw) look themselves up by the same string. DomainSet and CidrSet have private fields, so outside the module build_geo_data is the only way to fill them; a consumer can still pass GeoData::default(). A code that is referenced but absent from the map, which only happens when a consumer builds RouteMatch values without passing them through build_geo_data, never matches.
DomainSet::matches is an OR over its matchers. CidrSet::matches is self.cidrs.iter().any(|c| c.contains(&ip)) ^ self.reverse, so a reversed set matches every address outside the entry, IPv6 addresses included when the entry lists only IPv4 ranges.
The .dat messages
Section titled “The .dat messages”The protobuf messages are public structs deriving prost::Message, and they are deliberately minimal: prost skips fields a message does not declare, so extra fields in newer .dat files are ignored.
| Message | Field | Tag | Type |
|---|---|---|---|
GeoIpList |
entry |
1 | repeated GeoIp |
GeoIp |
code |
1 | string |
cidr |
2 | repeated Cidr |
|
Cidr |
ip |
1 | bytes, 4 or 16 of them |
prefix |
2 | uint32 |
|
GeoSiteList |
entry |
1 | repeated GeoSite |
GeoSite |
code |
1 | string |
domain |
2 | repeated Domain |
|
Domain |
type (r#type) |
1 | int32 |
value |
2 | string |
|
attribute |
3 | repeated Attribute |
|
Attribute |
key |
1 | string; the attribute’s value is not declared |
The message GeoIp and the variant RouteMatch::GeoIp share a name but are unrelated types.
A geosite Domain.type becomes one private DomainMatcher:
type |
DomainMatcher |
Matches when |
|---|---|---|
0 |
Substr |
the domain contains the value |
1 |
Regex |
the compiled value matches the domain |
2 |
Domain |
is_subdomain_of(domain, value) |
3 |
Full |
the domain equals the value |
| any other | none | the entry is skipped with a warning |
build_geo_data
Section titled “build_geo_data”pub fn build_geo_data( geoip_path: Option<&Path>, geosite_path: Option<&Path>, geosite_refs: &[CompactString], geoip_refs: &[CompactString],) -> io::Result<GeoData>;The consumer collects every geosite and geoip string its rules use and passes them as geosite_refs and geoip_refs. Geosite is handled first, then geoip; for each kind:
- If the reference list is empty, the file is not opened, even when a path is configured. A config that names a missing
geoipfile but uses nogeoipmatcher passes--test. - Otherwise the path must be configured, or the call fails with
a geosite matcher is used but no geosite file is configured(or thegeoipequivalent). - The whole file is read with
std::fs::readand decoded once into aGeoSiteListorGeoIpList. - Each raw reference not already in the map is resolved; a repeated raw string is skipped.
- geosite:
code@attris split on the first@. The entry is found bycodewitheq_ignore_ascii_case. With an attribute, only the domains that carry anAttributewhosekeyequals it, ignoring ASCII case, are kept. A leading!has no meaning here:"!cn"is looked up literally and fails as not found. - geoip: a leading
!setsreverseand is stripped. The entry is found by the remaining code, ignoring ASCII case.
- geosite:
- The decoded list is dropped once the references of its kind are resolved, so only the sets for referenced codes stay in memory.
Codes are looked up case-insensitively, but the map key is the raw string. "cn" and "CN" in two rules resolve to the same entry and are built twice, under two keys.
The two kinds treat a bad entry differently:
- Geosite entries are skipped with a
tracing::warn!: a regex that does not compile (skipping invalid geosite regex {value:?}: {e}) and an unknown type (skipping unknown geosite domain type {other}). One bad line in a community list does not take the whole code down. - Geoip entries are hard errors, because
build_cidr_setvalidates every CIDR:- the IP must be 4 or 16 bytes (
geoip cidr: ip must be 4 or 16 bytes, got {n}); - the prefix must fit a
u8(geoip cidr: prefix too large); IpCidr::newmust accept the pair, which rejects a prefix longer than the address and any set host bit (geoip cidr: {e}).
- the IP must be 4 or 16 bytes (
How the consumers compile their configs
Section titled “How the consumers compile their configs”Both consumers expose the same pair of functions: build_router compiles a route config against a map of built outbounds, and route_target adapts one flow to a RouteTarget.
pub fn build_router( cfg: &crate::config::RouteConfig, outbounds: &HashMap<CompactString, Arc<Outbound>>, default_tag: &CompactString,) -> io::Result<Router>;
pub fn route_target<'a>(flow: &'a Flow, ctx: &'a FlowContext) -> routing::RouteTarget<'a>;app/src/instance.rs → build calls build_router for a start, a reload and the --test dry run alike. default_tag is the tag of the first [[outbound]] (a config without one fails earlier with config defines no outbounds); [route].default overrides it. The outbounds map already contains every [[balancer]] as an Outbound::Balanced, so a rule or the default may name a balancer; the balancer picks a healthy member when the flow connects, not when it is routed.
route_target supplies every context field:
- the sniffed domain from
flow.sniffed; - the inbound tag from
FlowContext::inbound_tag; - the source as
flow.source.or(ctx.source), so a flow’s own source (a QUIC client, a TUN host) wins over the peer the listener captured at accept; - the network from
flow.destination.network.
FlowContext is built once per accepted connection in app/src/serve.rs, and per peer by the Hysteria 2 and TUN inbounds with source: Some(ip).
pub fn build_router<Outbound>( cfg: &crate::config::RouteConfig, outbounds: &HashMap<CompactString, Arc<Outbound>>, default_tag: &CompactString,) -> io::Result<Router<Outbound>>;
pub fn route_target<'a>( dest: &'a Destination, sniffed: Option<&'a SniffedBehavior>, source: Option<IpAddr>,) -> routing::RouteTarget<'a>;build_router is generic over the outbound type. katana calls it with default_tag "direct" from four places:
NodeManager::new, for every node at start and, throughbuild_nodeinapply_reload’s pre-build, for every node a reload adds;NodeManager::apply_static, for a[node.route]edit to a running node;NodeManager::rebuild_router, after an[[outbound]]pool change;test_configinsrc/runtime.rs, for the--testdry run.
build_outbounds always seeds the pool with the reserved tags direct and freedom (both a FreedomConnector) and block and blackhole (both Outbound::Block), and refuses a configured outbound that reuses one of them.
route_target supplies the sniffed domain, the source and the network. katana never supplies an inbound tag, and its config has no key that would produce an InboundTag matcher.
Each rule field becomes matchers in a fixed order. Because a rule is an OR, the order inside one rule does not change the result, only which matcher short-circuits first.
| Config key | RouteMatch |
Compiled with | app | katana |
|---|---|---|---|---|
domain_suffix |
DomainSuffix |
to_ascii_lowercase |
yes | yes |
domain_keyword |
DomainKeyword |
to_ascii_lowercase |
yes | no |
domain_full |
DomainFull |
to_ascii_lowercase |
yes | no |
domain_regex |
DomainRegex |
routing::parse_domain_regex |
yes | no |
cidr |
Cidr |
str::parse::<IpCidr> |
yes | yes |
source_cidr |
SourceCidr |
str::parse::<IpCidr> |
yes | no |
inbound_tag |
InboundTag |
as written | yes | no |
network |
Network |
parse_network: "tcp" or "udp" only |
yes | no |
port |
PortRange |
parse_port_match |
yes | yes |
geosite |
GeoSite |
also pushed to geosite_refs |
yes | yes |
geoip |
GeoIp |
also pushed to geoip_refs |
yes | yes |
RouteConfig and RuleConfig carry #[serde(deny_unknown_fields)] in both consumers. A katana rule with domain_keyword therefore fails to parse (unknown field `domain_keyword`, expected one of `outbound`, `domain_suffix`, `cidr`, `port`, `geosite`, `geoip` ) instead of being ignored.
A few consequences of the compile step that are easy to miss:
cidrandsource_cidrvalues go through thecidrcrate, which rejects set host bits:"192.0.2.1/24"fails withinvalid cidr "192.0.2.1/24": host part of address was not zero. A value without a prefix, such as"192.0.2.1", parses as a single host (/32, or/128for IPv6).networkis one string, not a list, so a rule can require at most one transport;"tcp,udp"fails withinvalid rule network "tcp,udp" (expected "tcp" or "udp").inbound_tagvalues are not checked against the configured inbounds. A tag that names no inbound makes an inert matcher.- Geodata is loaded after every rule has compiled and every outbound tag has resolved, so an unknown tag or a bad port is reported before the
.datfiles are read.
Where the router lives
Section titled “Where the router lives”| etemenanki-app | katana | |
|---|---|---|
| Held as | Arc<Router> in Built, cloned into every AppConnector and FanOutLink |
parking_lot::Mutex<Arc<Router<Outbound>>> in NodeManager, and an Arc<Router<Outbound>> in each listener generation’s Dispatcher |
| Rebuilt when | a reload builds a new generation | the node’s route config or the global outbound pool changes |
| Build failure at start | the process exits after failed to start: {e} |
spawn_node logs node {id}: build router: {e} and skips that node; the other nodes start |
| Rebuild failure | reload logs reload: build failed, keeping current config: {e} and the old generation keeps serving, see Generations and reload |
depends on what triggered the build; see the list below |
| Geodata | decoded once per generation | decoded separately for each node, and again on every rebuild |
When a katana router does not compile after start, the outcome depends on what triggered the build:
- A node added by a reload.
apply_reloadbuilds every node the reload adds, router included, before it touches any running node. A node counts as added when no running node has its identity: a new[[node]]entry, one whose identity changed, or one that failed to build at start, so such an entry is built again on every reload, and every reload is refused until it builds or is removed. If one fails, it logsreload: node {display_id}: build router: {e}; keeping current configand applies nothing from the new config: the log level, the outbound pool and every node stay as they were.{display_id}is<panel_type>@<host>#<node_id>with the panel type lowercased, followed on newV2board by/<type>, the node type UniProxy is asked for (the lowercasednode_type, orvlesswhenenable_vlessis set on a V2ray-family node), because newV2board finds a node by its type as well as its id. See Process runtime and reload. - A
[node.route]edit.apply_staticcompiles the new route against the current pool before it stores any part of the edit. If that fails, it logsnode {id}: config edit refused, keeping the running one: {e}and refuses the node’s whole edit, including any other field changed alongside the route. The running router and transport stay, so no connection drops. An edit that compiles replaces the router and, once the node is up, forces one panel poll, which rebuilds the node’s transport. A node still coming up builds its first listener with the new router on its next attempt. - An
[[outbound]]pool change.rebuild_routerrecompiles the node’s unchanged route against the new pool. If that fails, it logsnode {id}: route rebuild failed, keeping current: {e}and keeps the previous router. The node’s transport is still rebuilt afterwards, so live connections drop even though the routing stays the same; see Node manager.
The fan-outs use the outbound’s Arc pointer as its identity. That pointer is stable because the router holds its outbounds for its whole lifetime.
Per-packet UDP routing
Section titled “Per-packet UDP routing”A UDP association has no single destination: every datagram carries its own. Routing the association as a unit would send every peer’s traffic to one outbound and exempt UDP from the route rules. Both consumers therefore turn a UDP flow into a fan-out link instead of dialing it:
- the app:
AppConnector::connectreturnsOutbound::Datagram(FanOutLink::new(router, ctx, flow))whenflow.destination.network == DialNetwork::Udp; - katana:
KatanaConnector::connectadmits the user, then returns aFanOutunder the same test.
Every poll_send_to routes the packet’s own destination:
- The app builds the target from
self.flow.toward(to.clone()).Flow::towardkeeps the user and source but setssniffedtoNone; theFlowContextstill supplies the inbound tag. - katana calls
route_target(to, None, self.source), with the source fixed when the association was admitted.
Per-packet routing never sees a sniffed domain: a domain rule matches a UDP packet only when the packet itself is addressed by name.
sequenceDiagram
participant RT as Connection runtime
participant F as FanOutLink
participant R as Router
participant O as Outbound
participant S as Sub-link
RT->>F: poll_send_to(buf, to)
F->>R: route(route_target(flow.toward(to), ctx))
R-->>F: Arc of outbound, key = Arc::as_ptr
alt a sub with this key exists
F->>S: poll_send_to(buf, to)
Note over F: on Ok, move the sub to the back
else nothing is opening
F->>O: connect_datagram(flow.toward(to))
Note over F: opening = Some((key, future)), then loop
else this key is opening
F->>F: poll_opening, then send, or drop the packet if the open failed
else another key is opening
F->>F: poll_opening, Pending until it resolves
end
RT->>F: poll_recv_from(buf)
F->>F: poll_opening, to progress a dial in flight
F->>S: poll_recv_from, round robin from next
S-->>RT: payload and from
The fan-out keeps these fields (katana’s FanOut adds disp, gate, uid and source):
| Field | Type | Role |
|---|---|---|
subs |
Vec<Sub> |
one sub-link per outbound; the most recently sent to is at the back |
next |
usize |
where the last receive stopped, so every sub gets a turn |
opening |
Option<(usize, OpenFuture)> |
at most one outbound being opened, with its key (DatagramFuture in katana) |
recv_waker |
Option<Waker> |
the parked receiver, woken when a new sub joins |
Behaviour to keep in mind before changing it:
- Sub-links are per outbound, not per peer. The key is
Arc::as_ptr(outbound) as usize, so every peer routed to the same outbound shares one sub-link, and a balancer counts as one outbound. MAX_SUBS = 64. When a new sub pushes the table past 64, the least recently sent to (subs[0]) is dropped andnextmoves down by one.- One open at a time. A packet for another outbound returns
Pendingwhile an open is in flight, and the runtime keeps it queued. - A failed open drops the packet.
poll_send_toreturnsOk(buf.len()), so the association does not stall on an outbound that cannot be reached, and the failure is logged atdebugasudp fan-out: opening an outbound failed: {e}. In the app this includes outbounds that carry no datagrams:http,shadowsocksandshadowsocks-2022failconnect_datagramwith{what} carries no datagrams. - A send error from an open sub-link is returned as is; only a failed open is turned into a silent drop.
- A sub-link that errors on receive is removed, logged at
debugasudp fan-out: a sub-link ended: {e}, and the receiver re-wakes itself to poll the rest.
In the app, a blackhole outbound gets a real sub-link (BlackholeLink) that swallows packets. katana short-circuits instead: when the route picks Outbound::Block, or the audit rules forbid the destination, poll_send_to returns Ok(buf.len()) before the user’s gate is polled, so the packet is dropped and not billed. katana’s billing and audit side of FanOut is covered in Connector and UDP.
Invariants
Section titled “Invariants”| Invariant | Enforced by | Pinned by |
|---|---|---|
| The first matching rule wins; later rules are not consulted after a hit. | the early return in RouteTable::pick |
first_matching_rule_wins |
| A rule matches when any one of its matchers matches. | item.matchers.iter().any(...) |
any_matcher_within_a_rule_matches |
| With no hit, the default is returned. | the final self.default.clone() |
default_fallback_when_no_rule_matches |
| A matcher whose field was not supplied does not match. | Option comparisons and is_some_and in matches |
source_cidr_matches_the_client_address, network_matches_the_declared_transport, inbound_tag_matches_the_declared_inbound |
| Domain matchers never match an IP target unless a sniffed domain is supplied. | Domains::of yields None for an IP remote |
domain_matchers_never_match_ip_dests, a_sniffed_domain_makes_an_ip_target_match_domain_rules |
| Geosite also sees the sniffed domain. | GeoSite goes through domains.any |
a_sniffed_domain_feeds_geosite_too |
Cidr and GeoIp never match a domain target. |
the Remote::Domain(_) => false arms |
cidr_matches_the_destination_only (covers Cidr only) |
| Domain comparison ignores ASCII case for lowercase rules. | Domains::of folds the target; consumers fold the rule side |
domain_matching_is_case_insensitive |
| A suffix matches only on a label boundary. | is_subdomain_of |
domain_suffix_matches_parents_not_lookalikes |
| Port ranges are inclusive; an inverted, non-numeric or out-of-range spec is rejected. | parse_port_match |
port_ranges_are_inclusive, port_parser_rejects_an_inverted_range |
| An invalid regex fails the table build instead of becoming a dead rule. | DomainRegex::new, the only constructor |
invalid_domain_regex_is_rejected, domain_regex_matches_its_pattern |
| Two regex matchers are equal when their patterns are. | impl PartialEq for DomainRegex |
domain_regex_equality_is_by_pattern |
| Every context field the app’s config accepts reaches the router. | app route_target |
inbound_tag_selects_the_route, source_cidr_matches_the_client_address, network_separates_tcp_from_udp in app/tests/integration/e2e_route_context.rs |
| A flow’s own source wins over the listener’s. | flow.source.or(ctx.source) |
the_circuits_own_source_wins_over_the_listeners and its three siblings in app/tests/unit/router.rs |
| A sniffed domain changes the route end to end. | both route_target functions pass flow.sniffed |
e2e_sniff.rs in the app, katana tests/integration/sniff.rs |
| UDP is routed per packet, not per association. | FanOutLink, FanOut |
one_association_routes_each_peer_separately, replies_from_several_peers_merge_back_correctly, katana udp_is_billed_after_routing_and_blocked_packets_are_free |
| A referenced geo code must exist, and a used geo kind must have a readable file. | build_geo_data returns NotFound, InvalidInput or the std::fs::read error |
no unit test; checked by the --test dry run |
Unit tests listed without a file live in environment/tests/unit/routing.rs. That file is compiled into environment/src/routing.rs as mod tests through #[path = "../tests/unit/routing.rs"], so it can build private types such as DomainSet and DomainMatcher directly.
Failure paths
Section titled “Failure paths”pick and route are infallible. Every failure in this module happens while a table is built, and every one fails the whole build, so a bad rule is rejected instead of producing a table with a silently dead rule. What the rejection costs depends on the consumer: see Where the router lives.
| Condition | io::ErrorKind |
Message |
|---|---|---|
Port spec is not a u16 or a range of them |
InvalidInput |
invalid port spec: "70000" |
| Inverted port range | InvalidInput |
invalid port spec: "2000-1000" has a lower bound above its upper bound |
| Regex does not compile | InvalidInput |
invalid domain regex "(unclosed": regex parse error: … (multi-line) |
| Geosite matcher without a geosite file | InvalidInput |
a geosite matcher is used but no geosite file is configured |
| Geoip matcher without a geoip file | InvalidInput |
a geoip matcher is used but no geoip file is configured |
.dat file cannot be read |
from std::fs::read |
the OS error alone, for example No such file or directory (os error 2); the path is not included |
.dat file does not decode |
InvalidData |
geosite decode: … or geoip decode: … |
| Code not in the file | NotFound |
geosite code not found: {code} or geoip code not found: {code}; the geosite code is shown without its @attr, the geoip code without its ! |
| Malformed geoip CIDR | InvalidData |
geoip cidr: … |
| Unknown outbound tag in a rule or the default (consumers) | InvalidInput |
route references unknown outbound tag: {tag} |
Bad cidr or source_cidr (consumers) |
InvalidInput |
invalid cidr {s:?}: {e} |
Bad network (app) |
InvalidInput |
invalid rule network {other:?} (expected "tcp" or "udp") |
Under --test, the app prints these after configuration invalid: and katana after configuration error:. At run time they appear inside the start, reload and rebuild messages listed in Where the router lives. A geoip.dat configured as the geosite file fails with a geosite decode error rather than loading as wrong data.
There is nothing to cancel in the module itself. Connectors and fan-outs hold an Arc of the router they were created with, and neither consumer swaps a router under a live connection: the app’s reload builds the new generation first and then cancels the old one, which aborts its in-flight tasks, and katana rebuilds the node’s whole transport after a route change, which drops every connection on the node (pinned by route_change_drops_connections in katana tests/unit/e2e.rs). A flow is therefore routed by one router for its whole life.
Limits
Section titled “Limits”| Item | Value | Where |
|---|---|---|
| Matchers stored inline per rule | 3 | RouteItem::matchers, SmallVec<[RouteMatch; 3]> |
| Sub-links per UDP association | 64 | MAX_SUBS in app/src/outbound/udp_fanout.rs and katana src/connector.rs |
| Outbound opens in flight per association | 1 | FanOutLink::opening, FanOut::opening |
Allocations per pick |
up to two Strings, the folded request and sniffed domains, made even when the table has no domain rule |
Domains::of |
| Rule lookup | linear in the number of rules, and linear in the entries of each geosite or geoip set it reaches | pick, DomainSet::matches, CidrSet::matches |
| Geodata file | read whole into memory and decoded once per build; only referenced sets survive | build_geo_data |
For UDP, pick runs once per datagram, so its per-call cost is paid per packet.
| Test | File | Behaviour |
|---|---|---|
any_matcher_within_a_rule_matches |
environment/tests/unit/routing.rs |
a rule is an OR of its matchers |
first_matching_rule_wins |
same | order is priority |
default_fallback_when_no_rule_matches |
same | an empty table returns the default |
domain_suffix_matches_parents_not_lookalikes |
same | label-boundary suffix matching |
domain_keyword_matches_any_substring |
same | substring matching |
domain_full_matches_only_the_exact_name |
same | exact matching |
domain_regex_matches_its_pattern |
same | regex matching through parse_domain_regex |
invalid_domain_regex_is_rejected |
same | build-time regex validation |
domain_regex_equality_is_by_pattern |
same | DomainRegex equality and as_str |
domain_matching_is_case_insensitive |
same | target domains are folded |
domain_matchers_never_match_ip_dests |
same | an IP target has no domain |
a_sniffed_domain_makes_an_ip_target_match_domain_rules |
same | the sniffed domain reaches domain rules |
a_sniffed_domain_feeds_geosite_too |
same | the sniffed domain reaches geosite |
cidr_matches_the_destination_only |
same | Cidr reads IP remotes only |
source_cidr_matches_the_client_address |
same | the source is separate from the destination and inert when absent |
port_ranges_are_inclusive |
same | inclusive bounds |
port_parser_rejects_an_inverted_range |
same | inverted, non-numeric and out-of-range specs are rejected |
network_matches_the_declared_transport |
same | the network matcher, inert without context |
inbound_tag_matches_the_declared_inbound |
same | the inbound-tag matcher, inert without context |
the_circuits_own_source_wins_over_the_listeners, without_one_the_listeners_source_is_used, with_neither_there_is_no_source_to_match_on, a_circuit_source_works_with_no_listener_source |
app/tests/unit/router.rs |
source precedence in the app’s route_target |
route_rules_accept_every_matcher |
app/tests/unit/config.rs |
every matcher key parses into its own field |
inbound_tag_selects_the_route, source_cidr_matches_the_client_address, network_separates_tcp_from_udp |
app/tests/integration/e2e_route_context.rs |
the context matchers work end to end |
an_http_host_routes_an_ip_addressed_flow, a_tls_sni_routes_an_ip_addressed_flow, a_flow_whose_sniffed_host_does_not_match_is_blocked, an_unsniffable_payload_falls_through_to_the_default, turning_sniffing_off_stops_the_domain_rule_matching |
app/tests/integration/e2e_sniff.rs |
sniffed domains change the route end to end |
one_association_routes_each_peer_separately, replies_from_several_peers_merge_back_correctly |
app/tests/integration/e2e_udp_route.rs |
per-packet UDP routing and reply merging |
a_blocked_destination_is_refused |
katana tests/unit/connector.rs |
a TCP flow routed to block is refused |
udp_is_billed_after_routing_and_blocked_packets_are_free |
katana tests/unit/connector.rs |
per-packet routing; blocked packets are dropped unbilled |
a_sniffed_host_reaches_a_domain_rule, disable_sniffing_stops_the_domain_rule_matching |
katana tests/integration/sniff.rs |
katana passes the sniffed domain to the router |
route_change_drops_connections |
katana tests/unit/e2e.rs |
a route edit rebuilds the node and drops live connections |
build_geo_data, build_domain_set and build_cidr_set have no unit tests at the pinned revision. After changing them, the quickest check is the app’s --test dry run against real geoip.dat and geosite.dat files, which exercises decoding, code@attr, !code and the missing-code error. The routing unit tests run with cargo test -p etemenanki-environment; see Testing for the rest of the suite.