Skip to content

HTTP proxy

Source files: 19 · checked against Etemenanki 596916d · katana v3.0.1
  • Etemenanki/protocols/src/http/mod.rs
  • Etemenanki/protocols/src/http/config.rs
  • Etemenanki/protocols/src/http/protocol.rs
  • Etemenanki/protocols/src/http/core.rs
  • Etemenanki/protocols/src/http/codec.rs
  • Etemenanki/protocols/src/core/mod.rs
  • Etemenanki/protocols/src/sniff/mod.rs
  • Etemenanki/protocols/src/helpers/address.rs
  • Etemenanki/concepts/src/runtime.rs
  • Etemenanki/concepts/src/client.rs
  • Etemenanki/protocols/tests/unit/http/core.rs
  • Etemenanki/protocols/tests/unit/http/protocol.rs
  • Etemenanki/protocols/tests/unit/http/codec.rs
  • Etemenanki/protocols/tests/pipeline/http.rs
  • Etemenanki/app/src/config.rs
  • Etemenanki/app/src/inbound/mod.rs
  • Etemenanki/app/src/serve.rs
  • Etemenanki/app/src/outbound/mod.rs
  • katana/src/outbound/mod.rs

The HTTP proxy lives in protocols/src/http/. It has two halves that share one set of wire helpers:

  • HttpCore, the server side, is a ProxyCoreDecode that serves one HTTP/1.x proxy request per connection: a CONNECT tunnel, or a plain request with an absolute URI that it rewrites and forwards to the origin.
  • HttpConnect, the client side, is a ProxyCoreEncode codec that sends one CONNECT to an upstream proxy, waits for a 200, and then passes bytes through unchanged.

This page is for contributors who change either half. It assumes you know the server-core contract (events, effects, the held buffer and the staging reserve) from Server cores, and the shared Timing, SniffPrefix and Passthrough helpers from Protocol foundations. The user-facing settings are on the HTTP proxy guide page.

Component Does Leaves to others
protocol.rs Finds and parses request and response heads, holds the fixed responses, rewrites a forwarded request, builds a CONNECT request, checks Proxy-Authorization. All I/O. The helpers are pure functions over byte slices, except the unused async read_head.
HttpCore Parses the first request head, authenticates it, opens exactly one outbound flow, answers CONNECT, stages the fixed error responses, relays both directions verbatim afterwards. Dialing, routing, reading and writing sockets, and the watchdog for a client that never sends a byte (the runtime and the application).
HttpConnect Stages one CONNECT request, parses the upstream’s status line, then seals and opens bytes verbatim. Dialing the upstream proxy and any TLS around it (the client runtime and the outbound’s transport). UDP: HTTP proxies carry none.
HttpServerConfig Holds the account map, the anonymous payload and allow_transparent. Parsing TOML. The app builds it from HttpInboundSettings in app/src/inbound/mod.rs.

The helpers are a port of the HTTP proxy in Xray-core (proxy/http and common/protocol/http/headers.go); the sans-I/O core and codec around them are native to this crate.

protocols/src/http/config.rs → HttpServerConfig:

pub struct HttpServerConfig<T> {
pub accounts: HashMap<CompactString, (CompactString, Arc<T>)>,
pub anonymous: Arc<T>,
pub allow_transparent: bool,
}
Field Meaning
accounts username -> (password, payload). An empty map turns authentication off.
anonymous The payload every flow carries when accounts is empty. It is ignored otherwise.
allow_transparent Accept origin-form request targets (GET /path) and take the destination from Host. Default sets it to false.

Clone is implemented by hand so that T needs no Clone bound (every payload sits behind an Arc). Default requires T: Default, because it builds the anonymous payload with T::default(). The core receives the config as Arc<HttpServerConfig<T>>, so one config is shared by every connection of a listener.

The app maps its settings one to one: each [[inbound.settings.accounts]] entry (Account { user, pass }) becomes a map entry with payload Arc::new(()), and allow_transparent is copied from HttpInboundSettings, which is #[serde(deny_unknown_fields, default)].

protocols/src/http/core.rs → HttpCore:

pub struct HttpCore<T> { /* private */ }
impl<T> HttpCore<T> {
pub const BUF_SIZE: usize = MAX_HEAD;
pub fn new(config: Arc<HttpServerConfig<T>>, sniff: bool, source: Option<IpAddr>) -> Self;
pub fn is_established(&self) -> bool;
}
impl<T: Send + Sync + 'static> ProxyCoreDecode for HttpCore<T> {
type Key = Single;
type Target = Flow<T>;
type Error = io::Error;
type TransportAddr = ();
const STAGING_RESERVE: usize = 256;
fn handle(
&mut self,
event: Event<'_, Self>,
fx: &mut Effects<'_, Self>,
) -> Result<usize, io::Error>;
fn held(&self) -> &[u8];
}

The private fields tell you where every piece of state lives:

Field Type Role
config Arc<HttpServerConfig<T>> Accounts and allow_transparent.
sniff bool Whether the inbound has sniffing on (the app passes the inbound’s sniffing setting, which defaults to true). Only a CONNECT whose target is an IP literal sniffs.
source Option<IpAddr> The client address, copied into every Flow for routing.
timing Timing The single deadline, armed per phase. is_established() reads it.
prefix SniffPrefix The first tunnel bytes of a sniffing CONNECT, held until the flow opens.
rewritten Vec<u8> The origin-form head of a forwarded plain request.
state State<T> The state machine below.
enum State<T> {
Handshake,
Sniff(Flow<T>),
Relay {
relay: Passthrough<Single>,
reply: bool,
},
Done,
}

reply is true only for a CONNECT whose 200 is still owed: the core stages it on Connected. A sniffing CONNECT and a plain request both enter Relay with reply: false.

held() returns rewritten when it is non-empty and the sniff prefix otherwise. At most one of the two is ever filled on a connection, because a plain request never sniffs and a CONNECT never rewrites.

BUF_SIZE is the read-buffer size a runtime over this core needs. The app instantiates it as drive::<{ HttpCore::<()>::BUF_SIZE }, _, _> in app/src/serve.rs, and the pipeline tests use the same constant.

protocols/src/http/codec.rs → HttpConnect:

pub struct HttpConnect { /* private: request: Vec<u8> */ }
impl HttpConnect {
pub const REQUEST_MAX: usize = 1024;
pub fn new(dest: &Destination, auth: Option<(&str, &str)>) -> Self;
}
impl ProxyCoreEncodeHandshake for HttpConnect {
type Target = Destination;
type Error = io::Error;
const STAGING_RESERVE: usize = Self::REQUEST_MAX;
fn start(&mut self, out: &mut Staging<'_>) -> io::Result<Handshake>;
fn reply(&mut self, wire: &mut [u8], _: &mut Staging<'_>) -> io::Result<Reply>;
fn finish(&mut self, _: &mut Staging<'_>) -> io::Result<()>;
}
impl ProxyCoreEncode for HttpConnect {
fn seal(&mut self, plain: &[u8], out: &mut Staging<'_>) -> io::Result<usize>;
fn open(&mut self, wire: &mut [u8]) -> io::Result<Opened>;
}

new builds the whole request up front, so the codec’s only state is that byte vector, which start clears once staged. Both the app and katana wrap it as ProxyClient<HTTP_BUF, HttpConnect, NoUdp>; see Outbounds for the app and the katana outbound pool in Inbound and outbound.

protocols/src/http/protocol.rs:

pub struct Header {
pub name: String,
pub value: Vec<u8>,
}
pub struct RequestHead {
pub method: String,
pub target: String,
pub headers: Vec<Header>,
}
pub fn find_head_end(buf: &[u8]) -> Option<usize>;
pub fn parse_request_head(head: &[u8]) -> io::Result<RequestHead>;
pub fn parse_response_status(head: &[u8]) -> io::Result<u16>;
pub fn build_connect_request(dest: &Destination, auth: Option<(&str, &str)>) -> Vec<u8>;
pub fn header_value<'a>(headers: &'a [Header], name: &str) -> Option<&'a str>;
pub fn parse_request_target(target: &str) -> (bool, Option<&str>, &str);
pub fn build_forward_request(
method: &str,
origin_target: &str,
host: &str,
headers: &[Header],
) -> Vec<u8>;
pub fn check_proxy_auth<T>(
headers: &[Header],
accounts: &HashMap<CompactString, (CompactString, Arc<T>)>,
) -> Option<(String, Arc<T>)>;
pub async fn read_head<R>(reader: &mut R) -> io::Result<Vec<u8>>
where
R: AsyncRead + Unpin;
Helper Behaviour
find_head_end Offset just past the first \r\n\r\n, or None. It scans the whole slice each call.
parse_request_head httparse::Request over at most MAX_HEADERS (128) headers, copied into owned Headers. A partial parse is an error, so the caller must pass a complete head.
parse_response_status httparse::Response over the same header budget; returns only the status code.
header_value First header whose name matches case-insensitively, as UTF-8. A non-UTF-8 value reads as absent.
parse_request_target Splits a target into (is_https, authority, origin_form). The http:// and https:// prefixes match case-insensitively; the authority ends at the first /, ? or #; an absolute URI with nothing after the authority yields origin form /. Anything else returns (false, None, target).
build_forward_request Builds the origin-form head a plain request is forwarded as (see the forwarded head).
check_proxy_auth Verifies Proxy-Authorization: Basic … against the account map (see authentication).
read_head An async byte-at-a-time head reader capped at MAX_HEAD. Nothing in the workspace calls it; the core uses find_head_end on the runtime’s read buffer instead.

parse_authority and format_authority in protocols/src/helpers/address.rs convert between a host[:port] string and a TCP Destination. parse_authority(raw, default_port) trims surrounding whitespace, accepts a bracketed IPv6 literal with or without a port, takes default_port when the port is absent, keeps a domain unresolved, and rejects an unbracketed IPv6 literal, an invalid or empty port and an empty host. A host that does not parse as an IP address becomes a domain, including text inside brackets.

The server core never builds a response dynamically. It stages one of four constants from protocol.rs, all of which fit the 256-byte STAGING_RESERVE:

Constant Bytes Sent when Then
CONNECT_ESTABLISHED 39 A CONNECT target connected, or immediately for a sniffing CONNECT Relay (Sniff for a sniffing CONNECT)
RESP_407 106 Credentials are required and missing or wrong ShutdownTransport, Finish
RESP_400 72 A plain request is not absolute-form and allow_transparent is off ShutdownTransport, Finish
RESP_502 47 A CONNECT target failed to connect and the 200 is still owed ShutdownTransport, Finish
the four responses
HTTP/1.1 200 Connection established
HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: Basic realm="proxy"
Connection: close
HTTP/1.1 400 Bad Request
Proxy-Connection: close
Connection: close
HTTP/1.1 502 Bad Gateway
Connection: close

Every line ends in \r\n, and every response ends with an empty line and no body.

build_connect_request formats the target with format_authority (IPv6 in brackets, port always present) and writes:

Line Present Content
Request line always CONNECT <host>:<port> HTTP/1.1
Host always The same <host>:<port>.
Proxy-Authorization when auth is Some Basic followed by standard base64 of user:pass.
Proxy-Connection always Keep-Alive
Terminator always An empty line.

The request must fit REQUEST_MAX (1024 bytes): the authority twice plus one credential. start checks the length before staging and otherwise fails with InvalidInput and the text http: CONNECT request exceeds the codec's reserve.

For a plain request, build_forward_request produces this head, which the core keeps in rewritten:

Part Content
Request line <method> <origin-form target> HTTP/1.1. The method is copied; the version is always HTTP/1.1.
Host The authority from the absolute URI, or the client’s Host value when the URI had none (transparent mode).
Other headers Every client header in its original order, with its name spelled as the client sent it, written as name: value, except those dropped below.
Connection Always close, appended last.
Terminator An empty line.

Headers dropped from the client’s head:

  • every Host header (replaced by the line above);
  • every name in HOP_BY_HOP: proxy-connection, proxy-authenticate, proxy-authorization, te, trailers, transfer-encoding, upgrade, connection, keep-alive;
  • every name listed in the client’s Connection header values, compared in lower case.
stateDiagram-v2
  [*] --> Handshake
  Handshake --> Handshake: head incomplete, consume 0
  Handshake --> Relay: CONNECT, Open, reply owed
  Handshake --> Relay: plain request, Open and ForwardHeld
  Handshake --> Sniff: CONNECT to an IP with sniffing, 200 staged
  Sniff --> Relay: verdict, sniff deadline or client EOF
  Handshake --> Done: 407, 400 or client EOF
  Relay --> Done: ConnectFailed
  Done --> [*]

Timing runs alongside with its own phase: Handshake until the flow opens, Sniff while the prefix is collected, and Relay from the moment open pushes Effect::Open. That is why is_established() turns true at Open, before the outbound has connected. Relay ends through the Passthrough half-close bookkeeping or the idle deadline, both of which push Effect::Finish without changing state.

On Event::Transport in Handshake, on_transport:

  1. Calls find_head_end on the whole unparsed region. With no terminator it returns Ok(0), so the runtime keeps the bytes and reads more, unless the region has reached MAX_HEAD, which is an error.
  2. Parses exactly data[..end] with parse_request_head. Bytes after end (a request body, or tunnel bytes a client sent early) stay in the runtime’s buffer.
  3. Calls on_head, which authenticates first and then branches on the method, compared case-insensitively with CONNECT.
  4. Returns Ok(end), consuming only the head.

The target is authority-form and goes straight to parse_authority(&head.target, 443); the Host header is not consulted. What happens next depends on whether the core has to sniff.

The core opens the flow at once and owes the 200 until the outbound connects:

sequenceDiagram
  participant C as Client
  participant R as Server runtime
  participant H as HttpCore
  participant O as Outbound
  C->>R: CONNECT example.com:443 HTTP/1.1 and headers
  R->>H: Event Transport, whole head
  H->>R: Effect Open, reply owed, deadline RELAY_IDLE_TIMEOUT
  R->>O: connect through the router
  alt connected
    O-->>R: connected
    R->>H: Event Connected
    H->>R: stage CONNECT_ESTABLISHED
    R-->>C: HTTP/1.1 200 Connection established
    C->>R: tunnel bytes
    R->>H: Event Transport
    H->>R: Effect Forward, whole slice
    R->>O: bytes, verbatim
  else connect failed
    R->>H: Event ConnectFailed
    H->>R: stage RESP_502, ShutdownTransport, Finish
    R-->>C: HTTP/1.1 502 Bad Gateway
  end

Once in Relay, the core is a Passthrough<Single>: every transport slice becomes one Effect::Forward of the whole slice, and every outbound slice is staged verbatim toward the client.

For any other method, on_head resolves the destination and the forwarded Host separately:

flowchart TB
  target["parse_request_target(head.target)"]
  abs{"authority present and non-empty?"}
  transparent{"allow_transparent?"}
  r400["stage RESP_400, close"]
  host{"Host header non-empty?"}
  fromHost["destination from Host"]
  fromUri["destination from URI authority"]
  empty["error: missing target host"]
  open["parse_authority, build_forward_request, open"]
  target --> abs
  abs -- yes --> host
  abs -- no --> transparent
  transparent -- no --> r400
  transparent -- yes --> host
  host -- yes --> fromHost --> open
  host -- no --> fromUri
  fromUri -- "authority present" --> open
  fromUri -- "no authority" --> empty

The rules, precisely:

  • Destination. The Host header wins over the URI authority when it is non-empty. The default port is 443 for an https:// target and 80 otherwise.
  • Forwarded Host. The URI authority wins over the Host header. In transparent mode there is no authority, so the client’s Host is forwarded.
  • Forwarded target. The origin-form part of the URI (/path?q=1), or the target unchanged in transparent mode.

The core stores the rewritten head in rewritten and calls open(flow, false, fx), which pushes Effect::Open followed by Effect::ForwardHeld { range: 0..held }. The runtime applies the held forward once the outbound is connected, so the new head reaches the origin before any body byte:

sequenceDiagram
  participant C as Client
  participant R as Server runtime
  participant H as HttpCore
  participant O as Origin
  C->>R: GET http://example.com/path?q=1 HTTP/1.1, headers, body
  R->>H: Event Transport
  H->>H: build_forward_request into rewritten
  H->>R: Effect Open, ForwardHeld of rewritten, consumed = head only
  R->>O: connect, then GET /path?q=1 HTTP/1.1 ... Connection close
  R->>H: Event Transport, body bytes
  H->>H: clear rewritten
  H->>R: Effect Forward, whole slice
  R->>O: body, verbatim
  O-->>R: response, then EOF
  R->>H: Event Outbound, then OutboundEof
  H->>R: stage response, ShutdownTransport
  R-->>C: HTTP/1.1 200 OK ...

The core parses one head per connection. After it, everything the client sends is relayed to the same outbound without inspection, and the Connection: close the rewrite appends asks the origin to close after its response. A client that wants a second request has to open a new connection. The core does not stage a 200 for a plain request: the origin’s own response is the reply.

authenticate runs before the method branch, so it guards CONNECT and plain requests alike:

  • With an empty accounts map, every request is accepted as username "" with the anonymous payload.
  • Otherwise check_proxy_auth must return Some. It reads the first Proxy-Authorization header, strips a Basic or basic prefix (any other capitalisation of the scheme is refused), trims the rest, decodes standard base64, requires UTF-8, splits at the first : (so a password may contain colons, a username may not), looks up the username and requires the stored password to be equal. Any failure along that chain yields None.

A None sets State::Done and calls refuse(RESP_407, fx). On success the flow carries NetworkUser with UserAuthorization::UsernamePassword { username, password }, where password is always empty: the credential stops at the core. For plain requests the Proxy-Authorization header is also removed from the forwarded head by HOP_BY_HOP.

sequenceDiagram
  participant A as Client runtime
  participant X as HttpConnect
  participant U as Upstream proxy
  A->>X: start
  X->>A: stage CONNECT request, Handshake AwaitReply
  A->>U: CONNECT example.com:443 HTTP/1.1
  U-->>A: HTTP/1.1 200 ...
  A->>X: reply with the unparsed bytes
  alt no terminator yet
    X->>A: Reply NeedMore
  else status 200
    X->>A: Reply Step, consumed = head, next Done
  else any other status
    X->>A: error ConnectionRefused
  end
  A->>X: seal and open, verbatim
  • reply accepts exactly status 200; any other code, including other 2xx codes, is refused with ConnectionRefused and the text proxy responded with status <code>. The client runtime wraps every codec error as InvalidData but keeps its text, which is how the pipeline test sees 407 in the error from the plaintext side.
  • consumed is the head length only. Bytes the upstream sent after the head stay in the client runtime’s read buffer and become the first tunnel bytes.
  • seal stages the whole plaintext slice it is given and open returns the whole wire slice as one frame. finish stages nothing: the wire’s own EOF carries the half-close.
Invariant Enforced by Pinned by
Nothing is consumed until a whole head has arrived. on_transport returns Ok(0) while find_head_end is None. connect_to_a_domain_answers_200_once_connected (a 20-byte prefix consumes 0), head_end_is_found_only_once_the_blank_line_arrives
A request head always fits the read buffer, and a larger one fails in the core rather than stalling. BUF_SIZE = MAX_HEAD; on_transport errors when the region reaches MAX_HEAD with no terminator, before the runtime’s own frame-size check. an_oversized_head_is_refused
Only the head is consumed; a body or early tunnel bytes stay for the relay. on_transport returns end, not data.len(). plain_request_is_rewritten_into_the_held_buffer (consumed == wire.len() - 4)
No flow opens and no byte reaches an outbound before authentication. on_head calls authenticate first; failure goes to refuse(RESP_407, …). missing_or_wrong_credentials_are_a_407, new_server_refuses_bad_credentials_with_407
A CONNECT gets its 200 only after the target connected, unless it sniffs. State::Relay { reply: true }; Event::Connected stages CONNECT_ESTABLISHED and clears the flag. connect_to_a_domain_answers_200_once_connected
A CONNECT is answered exactly once. The sniffing path stages the 200 in on_head and opens with reply: false; Connected then stages nothing. connect_to_an_ip_with_sniffing_replies_early_and_holds_the_prefix (“no second 200”), new_server_connect_to_an_ip_answers_before_the_first_bytes, new_server_vs_new_client_tcp (a second 200 would corrupt the echoed bytes)
A failed CONNECT whose 200 is owed is answered 502 and closed. Event::ConnectFailed stages RESP_502 when reply is set, then ShutdownTransport and Finish. connect_refused_answers_502_and_closes
The rewritten head reaches the origin before the body. open pushes ForwardHeld right after Open; the runtime delivers no byte event until the held effect is applied. plain_request_is_rewritten_into_the_held_buffer, new_server_forwards_a_plain_request
The held buffer is never cleared under a queued held range. rewritten and prefix are cleared only at the start of a Relay byte event (Transport or Outbound), which the runtime’s pin rule delays until held effects are applied. plain_request_is_rewritten_into_the_held_buffer checks that held() is empty after the body; the pin rule itself belongs to the server runtime.
The rewritten head carries no proxy credentials and no hop-by-hop headers. build_forward_request skips HOP_BY_HOP names, names listed in Connection, and Host. forward_request_strips_hop_by_hop, plain_request_is_rewritten_into_the_held_buffer
A plain request without an absolute URI is refused unless transparent mode is on. on_head checks authority.is_none() && !allow_transparent. origin_form_without_transparent_is_a_400
Every fixed response fits the staging reserve. STAGING_RESERVE = 256; the largest response (RESP_407) is 106 bytes. A shortfall is a core bug and surfaces as staging_full(). Covered implicitly by every test that stages a response.
The codec’s request never exceeds its declared reserve. STAGING_RESERVE = REQUEST_MAX; start checks the length first. No test builds an oversized request.
Only a 200 opens a client tunnel, and the tail after it is preserved. reply matches 200 and returns consumed: end. connect_codec_refuses_a_non_200, connect_codec_awaits_a_200_then_passes_through

Every error that handle returns ends the connection through the runtime without a response to the client. The fixed responses cover the cases where the core has something useful to say.

Situation What the client sees Mechanism
Head reaches MAX_HEAD with no terminator Connection closed InvalidData, http head exceeds maximum size
Malformed head, or more than 128 headers Connection closed InvalidData, malformed http request: <httparse error>
Head made only of blank lines (for example a stray \r\n\r\n before the request) Connection closed InvalidData, incomplete http request head
Unparsable authority Connection closed parse_authority: invalid port, empty authority host, ambiguous authority (bracket IPv6 literals), malformed IPv6 authority, trailing data after IPv6 authority
Missing or wrong credentials 407, then close refuse(RESP_407, …), State::Done
Origin-form target, allow_transparent off 400, then close refuse(RESP_400, …), State::Done
Transparent mode, no Host and no authority Connection closed InvalidData, missing target host
CONNECT target fails to connect, 200 owed 502, then close Event::ConnectFailed stages RESP_502
Sniffing CONNECT or plain request fails to connect Connection closed, no response Event::ConnectFailed with reply: false: ShutdownTransport, Finish
Outbound errors during the relay Connection closed after staged bytes drain Passthrough::on_outbound_gone
Client closes before a complete head Connection closed TransportEof in Handshake pushes Finish
Client stalls after starting its head Connection closed Timing armed HANDSHAKE_TIMEOUT at the first byte; Expired::Handshake returns handshake_timed_out(): TimedOut, client did not complete its request in time
No bytes either way for RELAY_IDLE_TIMEOUT Connection closed Timing::expired pushes Finish (Expired::Idle)

refuse stages the response and then pushes Effect::ShutdownTransport and Effect::Finish, so the runtime writes the staged bytes before it closes the write side. ConnectFailed and OutboundError are logged at debug level as http: connect failed: … and http: outbound failed: ….

Half-closes follow Passthrough: client EOF becomes Effect::Shutdown on the outbound, outbound EOF becomes ShutdownTransport, and Finish follows once both halves have closed.

The core holds no task, lock or channel, so there is nothing of its own to cancel: when the runtime is dropped, the core goes with it. A client that connects and never sends a byte produces no event and so never arms the core’s deadline. The app’s drive in app/src/serve.rs covers that case by wrapping each runtime step in tokio::time::timeout(HANDSHAKE_TIMEOUT, …) until is_established() is true (failing with inbound handshake timed out after 10s), and releases the handshake permit at that point; see Serving.

On the client side, every handshake error fails the plaintext stream. The last two rows come from the client runtime, the others from the codec:

Situation Error
Request longer than REQUEST_MAX InvalidInput, http: CONNECT request exceeds the codec's reserve
Status other than 200 ConnectionRefused, proxy responded with status <code>
Response head that httparse rejects InvalidData, malformed proxy response: <httparse error>
Response head without a status line (only blank lines) InvalidData, proxy response missing status code
Response head with no terminator within MAX_HEAD InvalidData, http: proxy response head exceeds maximum size
Response head larger than the client runtime’s read buffer InvalidData, upstream frame larger than the client runtime's buffer
Upstream closes before a complete reply UnexpectedEof, upstream closed during the handshake

The client runtime re-wraps the codec’s errors as InvalidData and keeps their text. The app and katana size that runtime with HTTP_BUF (16 KiB), smaller than MAX_HEAD, so for them an oversized response head fails in the runtime before the codec’s own check can fire.

Constant Value Where Governs
MAX_HEAD 64 KiB (64 * 1024) protocols/src/http/protocol.rs Largest request head the core, or response head the codec, will buffer.
HttpCore::BUF_SIZE MAX_HEAD protocols/src/http/core.rs Size of each of the server runtime’s three buffers (transport read, transport staging, outbound scratch).
MAX_HEADERS 128 protocols/src/http/protocol.rs Header slots given to httparse, for requests and responses.
HttpCore STAGING_RESERVE 256 bytes protocols/src/http/core.rs Room guaranteed for a fixed response.
HttpConnect::REQUEST_MAX 1024 bytes protocols/src/http/codec.rs Largest CONNECT request, and the codec’s STAGING_RESERVE.
HANDSHAKE_TIMEOUT 10 s protocols/src/core/mod.rs From the first byte to a complete head.
SNIFF_TIMEOUT 300 ms protocols/src/sniff/mod.rs Sniffing window of a CONNECT to an IP.
SNIFF_LIMIT 4 KiB protocols/src/sniff/mod.rs Largest sniff prefix held.
RELAY_IDLE_TIMEOUT 300 s protocols/src/core/mod.rs Idle limit of a relay, refreshed on every Transport and Outbound event.
HTTP_BUF 16 KiB app/src/outbound/mod.rs, katana src/outbound/mod.rs Client runtime buffer of the HTTP outbound; bounds the upstream’s response head in practice.
Default ports 443 for CONNECT and https://; 80 otherwise protocols/src/http/core.rs Port used when the authority has none.

Allocation per connection is bounded by these numbers: rewritten is at most one head plus a request line, a Host line and Connection: close, and the sniff prefix is capped by SNIFF_LIMIT.

There are two kinds of tests. The unit tests in protocols/tests/unit/http/ are compiled into the crate through #[path] modules, so they can reach private items; the pipeline tests run the real runtimes over loopback TCP.

File Test Pins
protocols/tests/unit/http/core.rs connect_to_a_domain_answers_200_once_connected Partial heads consume nothing; a domain CONNECT opens immediately, stages nothing until Connected, and is established at Open.
connect_refused_answers_502_and_closes ConnectFailed stages RESP_502, then ShutdownTransport and Finish.
connect_to_an_ip_with_sniffing_replies_early_and_holds_the_prefix The early 200, the SNIFF_TIMEOUT deadline, the sniffed domain on Open, the ForwardHeld of the prefix, and no second 200.
plain_request_is_rewritten_into_the_held_buffer Origin-form rewrite, header stripping, Connection: close, head-only consumption, body forwarded after it, held buffer cleared.
origin_form_without_transparent_is_a_400 RESP_400 and Finish without opening.
missing_or_wrong_credentials_are_a_407 RESP_407 without credentials; a correct Basic header opens a flow as alice.
an_oversized_head_is_refused MAX_HEAD bytes without a terminator is an error.
protocols/tests/unit/http/protocol.rs target_absolute_form, target_absolute_form_no_path, target_origin_form parse_request_target splitting and the https flag.
forward_request_strips_hop_by_hop HOP_BY_HOP and Connection-listed headers are dropped.
head_end_is_found_only_once_the_blank_line_arrives find_head_end offsets.
request_head_is_parsed_into_owned_parts parse_request_head, header_value, check_proxy_auth, and rejection of garbage.
connect_request_and_response_status_round_trip build_connect_request parses back; parse_response_status reads 200, 407 and 502 from the fixed responses.
protocols/tests/unit/http/codec.rs connect_codec_awaits_a_200_then_passes_through The request line and credential, NeedMore on a partial head, the tail after the head, verbatim open and seal.
connect_codec_refuses_a_non_200 A 407 is ConnectionRefused.
protocols/tests/pipeline/http.rs new_server_vs_new_client_tcp HttpConnect against HttpCore with credentials, to an IP target with sniffing on (so the early-200 path): 100,000 bytes echoed, clean close.
new_server_refuses_bad_credentials_with_407 A wrong password surfaces as an error containing 407.
new_server_forwards_a_plain_request A real origin sees GET /path?q=1 HTTP/1.1 and the client gets its response.
new_server_connect_to_an_ip_answers_before_the_first_bytes With sniffing on, the 200 arrives before the client sends payload.

Run them from the Etemenanki workspace:

Terminal window
cargo test -p etemenanki-protocols --lib -- http::core http::protocol http::codec
cargo test -p etemenanki-protocols --test pipeline http::

A bare --lib http:: filter would also run the HTTP sniffer’s tests in sniff::http. The core tests drive HttpCore through CoreHarness, and the file’s moves helper filters SetDeadline effects out so assertions stay about behaviour. A change to an event path belongs in tests/unit/http/core.rs; a change to what a real client or origin sees on the wire also needs a pipeline test. Testing describes the harnesses.