Python SDK per-call overhead¶

Development A/B, not a standings run. Source: benchmarks/results/python_sdk_overhead.json, produced by bindings/python/holographic-sdk/benchmarks/client_overhead.py.

Setup: Apple M-series, 10 threads, CPython 3.14.7, httpx 0.28.1. Local release hms-server (--features security,server, default settings), 10,000 rows, dimension 384, top_k 10, sequential calls on one connection. One process per SDK build per round, 3 rounds with the build order rotated; within a process, raw http.client, the sync client and the async client alternate in 20 blocks with a rotating order, so the SDK-minus-raw difference is paired. 1-minute load average was 2.7 to 8.0 during the runs (other jobs shared the machine). Writes fsync on the server (about 4 ms), so their SDK difference is within noise; the p99 of the before writes reflects load spikes in its first round, not the client.

• before: SDK at 82b42f7b (httpx default transport, json= bodies; sniffio not installed).
• after: this change, stdlib json.
• after-orjson: this change with the fast extra (orjson).

p50 / p99 in microseconds; the last column is the query p50 minus raw's in the same process (median of the 3 rounds). Raw encodes its body with stdlib json.dumps.

Where the time went (cProfile and a timed transport comparison, 384-dim query):

• httpx's default transport (httpcore + h11 framing) cost about 70 us more per call than http.client; the sync client now uses a keep-alive http.client transport for plain-HTTP URLs with no proxy configured.
• httpcore imports sniffio on every async lock setup; without it installed, every async request repeated failed import lookups (about 20 path-finder searches). sniffio is now a dependency.
• Encoding 384 floats with stdlib json takes about 90 us, the same cost raw pays; orjson cuts it to under 10 us, which is why the fast build is below raw.
• The remaining async overhead is httpx/anyio scheduling (about 9 event-loop iterations per request).

The server has no batch-query or binary-vector route, so neither is in the SDK; both need a versioned server endpoint.