The proxy tier
nanocached-proxy is optional. A cluster works without it:
every SDK fetches the roster from discovery and talks to the nodes
directly, one connection per node per client process (architecture). The proxy exists for the
case where that stops scaling — a large or elastic application fleet,
where the node-side connection count (processes × nodes) grows past
what a node should hold — and for the case where the application side
can only be given one stable address. It terminates many client
connections, routes on their behalf with the same rendezvous hashing
the SDKs use, and keeps one pipelined connection per node, so the
node-side count becomes the proxy count. To a client it looks like a
single node that owns every key.
This page is about where to run it and how many, not how it works; the sidecar section is the alternative, with the constraints that make it the alternative.
When you need one
- Connection count. Every client process holds one
connection per node. With P processes and N
nodes, each node sees P connections; a node's own bound is
its
--max-connections. When P is in the thousands, or elastic enough that you cannot say what it is, put a proxy tier in front: each node then sees one connection per proxy.nanocached_node_connectionsagainstnanocached_node_connections_maxon the operations endpoint is the number to watch. - One address. A client that cannot run a discovery client of its own — a bare protocol client, or a fleet you would rather hand a single endpoint — points at a proxy (or a load balancer in front of several) and needs nothing else.
- Otherwise, no. A fixed fleet of a few dozen application servers does not need one. A cluster of plain hosts is that shape; if you do want a proxy there, run it the way you run a node — a systemd unit from the same image, on at least two hosts, one per zone.
Where it runs
The proxy is part of the cache cluster, not of the application: it is stateless, it registers itself with discovery, and it needs to reach discovery and the nodes — nothing about the application. So it runs with the cluster, in the cluster's own shape: a third service next to discovery and node in whatever orchestrator runs them, a third Deployment on Kubernetes. Applications on any platform in the same network reach it the same way. This keeps one operational model per cluster; a proxy tier on plain hosts in front of an orchestrated cluster would work, but it would mean images and manual replacement for one component and service definitions for the rest.
How applications find it. A proxy registers its own address
with discovery on start. An SDK in proxy mode (via_proxy
/ viaProxy / ViaProxy — see the SDK docs) is still configured with discovery's
address; it fetches the proxy roster instead of the node roster and
connects to one proxy, chosen at random, reconnecting to another if it
goes away. No service-discovery entry of the orchestrator's own is
needed for the proxies themselves. If you want a single address
instead, put an internal TCP load balancer in front of the proxies
and configure the SDK with that address as a plain single node.
How many. The three roles scale for three different reasons, which is why they are three services with three desired counts:
| role | count is set by | rule of thumb |
|---|---|---|
| discovery | nothing — it is the coordinator, joins are
staged one at a time through it, and a larger desired count on the
one service would not help: a second task behind the same name is
not a replica. Its loss stops new client bootstraps until the
scheduler replaces it; nodes and proxies keep working on their last
roster. If that gap matters, run a second discovery service
with its own name and list both, in the same order, in every
--discovery and every client's address list: a client
given both can bootstrap and read and write while the first is down
(joins still wait for the first-listed one). | 1 per service |
| node | data: dataset × R ÷ the process's
--max-memory, even across zones | ≥ 2, even |
| proxy | client connections and throughput: concurrent
application connections ÷ --max-connections (1024 per
proxy) with headroom; at least one per zone for availability. Node
count does not enter into it. A proxy also caps connections per
source IP (--max-connections-per-ip, 256 by default, the
node's own rule): when application tasks reach it through one NAT or
egress IP, that cap is the fleet's ceiling — raise it. | ≥ 2, one per zone;
grow when nanocached_proxy_client_connections approaches
_max |
For example, 40 application tasks × 8 connections = 320 client connections: two proxies (capacity 2,048) carry that, and growing the dataset changes the node count, not the proxy count.
The proxy mirrors the node's other per-connection limits: a request is at most 1 MiB, and a connection idle for 60 seconds is closed. The idle time is measured from the last complete frame, so a frame trickled in a byte at a time does not extend it — the node's own rule.
The sidecar alternative
A proxy container inside each application task or pod,
with the application configured for 127.0.0.1:8358 as a
plain single node — not proxy mode, which would pick a random
proxy from discovery's roster, most likely on another host. What it
buys: no extra network hop, and an application configuration that is
one fixed address. What it costs:
- Every sidecar registers with discovery like any proxy, so the proxy roster lists all of them — discovery caps it at 1,024 — and any client that does use proxy mode may be handed a sidecar on another host.
- Each sidecar holds its own connection to every node, so the node-side count is the application task count, not the proxy count: a reduction from processes to tasks, not to a handful.
- The sidecar's stop is the task's stop: the orchestrator's stop grace must still cover its drain, and it must reach discovery and the nodes from the application's network position.
Sensible for a small fleet that wants the single-address
configuration; not the way to bring the connection count down. Make
the application container, not the proxy, decide when the task ends
(a non-essential container in the task definition, or a second
container in the application pod on Kubernetes — or a DaemonSet
with hostPort and application pods pointed at
status.hostIP).