Getting started
A single node is one command. A replicated cluster is three more.
Run a single node
Docker
docker pull ghcr.io/nanocached/nanocached-node:latest
docker run --rm --publish 8356:8356 ghcr.io/nanocached/nanocached-node:latest
From source
Requires Rust (the repository pins the toolchain via
rust-toolchain.toml, so Rustup sets everything up):
git clone https://github.com/nanocached/nanocached.git
cd nanocached
cargo run --bin nanocached-node -- --host 0.0.0.0 --port 8356
The node listens on 127.0.0.1:8356 by default; override
with --host and --port. A release build
(cargo build --release) also produces ncd, a
thin dispatcher: ncd node start … /
ncd discovery start ….
Smoke test
printf 'S 4 5\nnameAliceG 4\nname' | nc 127.0.0.1 8356
Expected response: S, then V 5 /
Alice. See the protocol
reference for every command.
Build a cluster
A cluster is N cache nodes plus one or more discovery servers. The discovery server is a lightweight membership registry — it is never on the data path; clients fetch the node list from it and talk to nodes directly (how it works).
Start a discovery server:
docker run --rm --publish 8357:8357 ghcr.io/nanocached/nanocached-discovery:latest
# or: cargo run --bin nanocached-discovery -- --port 8357
Start nodes pointing at it:
cargo run --bin nanocached-node -- --port 8356 --discovery 127.0.0.1:8357
cargo run --bin nanocached-node -- --port 8358 --discovery 127.0.0.1:8357
Each node sends a heartbeat every 5 seconds; the discovery server
derives the node's reachable address from that connection's own source
IP plus the node's port, so containerized deployments need no address
configuration. (Asymmetric port mapping — Docker's
-p 9999:8356 — is unsupported: it is NAT, which the
cluster design already excludes.) New nodes join with a staged data
handoff, so adding a node never drops the keys it takes ownership
of.
Replication factor
Discovery owns the cluster's replication factor R (default 2, minimum 1):
cargo run --bin nanocached-discovery -- --port 8357 --replication-factor 2
Every key lives on its top-R nodes as ranked by rendezvous hashing, so
any single node death costs no cached data. Effective cluster capacity is
total memory ÷ R; --replication-factor 1 restores
single-copy behavior. Clients learn R automatically from the node list —
no SDK configuration.
Discovery replicas (no single point of failure)
The registry is soft state rebuilt from node announces, so discovery replicas run with zero coordination between them. Give every node — and every SDK client — the same list in the same order:
cargo run --bin nanocached-discovery -- --port 8357
cargo run --bin nanocached-discovery -- --port 8358
cargo run --bin nanocached-node -- --port 8356 \
--discovery 127.0.0.1:8357,127.0.0.1:8358
Losing any one replica — including the first-listed primary — costs
neither cache traffic nor client bootstrap; only joins need the
primary up. A restarted replica answers B (busy) for the
liveness-timeout window while live members re-announce, so a
bootstrapping client never sees a half-recovered node list.
Each replica needs its own stable, individually addressable
name (or a stable IP) — an operational requirement the system
does not verify. Every node and client must hold the same list in the
same order; a single DNS name that round-robins across several
replicas (or a multi-value service-discovery entry) breaks that,
since which replica a resolution lands
on — and in what order — becomes nondeterministic per resolver, per
lookup. A bare task/pod IP has the same problem in a different shape: it
works until that replica restarts with a new one, silently dropping it
from every --discovery list. Give each replica a stable
identity instead — on a container orchestrator, one service entry (or
load-balancer target group) per replica rather than one round-robin
service in front of all of them;
on Kubernetes, a headless Service-per-replica or a StatefulSet's stable
per-pod DNS names.
Every replica must be started with the same
--replication-factor — nothing enforces this at startup, since
replicas never talk to each other. A node reports the replication
factor it has learned on every heartbeat; while any joined node's
reported value disagrees with a replica's own configured value, that
replica logs a loud WARN and refuses node-list requests
with B (the same "retry elsewhere" answer as its startup
grace) rather than serving a replica count known to disagree with what
part of the cluster learned elsewhere. Clients simply skip to the next
seed, so a partially rolled-out misconfiguration degrades loudly
instead of silently splitting routing.
Authentication
Set NANOCACHED_AUTH_SECRET to require clients to
authenticate before any other command. It is an environment variable —
not a flag — so the secret never shows in ps output. Set it
on both roles to protect a cluster:
NANOCACHED_AUTH_SECRET=change-me nanocached-node --port 8356 --discovery 127.0.0.1:8357
NANOCACHED_AUTH_SECRET=change-me nanocached-discovery --port 8357
Nodes reuse the same value to authenticate their own heartbeats.
Unset (or empty) disables authentication entirely, and a server that
then listens on anything but a loopback address prints a one-line
WARN at startup (it still starts: a container has to listen
on 0.0.0.0). Set a secret, bind to 127.0.0.1, or
make sure the network (a security group, a NetworkPolicy, a firewall)
restricts who can connect. Authentication is a
second layer of defense, not a substitute for network isolation: without
TLS there is no transport encryption, so bind to 127.0.0.1
or a private interface and add TLS when the network is not trusted.
TLS
Pass --tls-cert/--tls-key (PEM) to require
TLS on every accepted connection — no plaintext fallback once set:
nanocached-node --port 8356 --tls-cert cert.pem --tls-key key.pem
nanocached-discovery --port 8357 --tls-cert cert.pem --tls-key key.pem
A node registering with a TLS-secured discovery server also needs
--tls-ca (the CA bundle to trust for its heartbeat
connection — only those CAs, not the system store, since cluster
certificates are typically private):
nanocached-node --port 8356 --tls-cert cert.pem --tls-key key.pem \
--tls-ca ca.pem --discovery 127.0.0.1:8357
For local development, generate a self-signed certificate that rustls
will accept as a leaf (CA:FALSE matters — OpenSSL's default
self-signed certs are marked as their own CA and get refused):
openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 1 \
-subj "/CN=127.0.0.1" \
-addext "subjectAltName=IP:127.0.0.1" \
-addext "basicConstraints=critical,CA:FALSE" \
-addext "keyUsage=critical,digitalSignature,keyEncipherment" \
-addext "extendedKeyUsage=serverAuth"
Being self-signed, the same cert.pem doubles as the
--tls-ca trust anchor for anything connecting to it. For
what each flag encrypts in a cluster, certificate names, distribution
and rotation, see TLS under
Deployment.
Connect a client
Use any of the six SDKs — each detects whether it reached a node or a discovery server from the handshake itself, so single-node development and clustered production use identical code. See SDKs.
Built-in limits
| Limit | nanocached-node | nanocached-discovery |
|---|---|---|
| Maximum request size | 1 MiB | 4 KiB |
| Maximum concurrent connections | 1,024 by default
(--max-connections; per source IP 256 by default,
--max-connections-per-ip — the effective ceiling behind
NAT or on Kubernetes, where clients share a source IP; an IPv6 source
counts per /64 prefix) | 1,024, and 256 from any one source IP
(both fixed; the --max-connections* flags don't exist here,
since discovery serves only brief roster fetches and heartbeats; an
IPv6 source counts per /64 prefix) |
| Cache memory bound | 256 MiB by default (--max-memory), LRU eviction;
per-namespace caps via --namespace-budget <ns>=<bytes>
(a namespace over its cap evicts from itself first) | — |
| In-flight bytes (replies being written, requests still arriving, the per-key working memory of multi-key frames) | 256 MiB
by default (--max-in-flight-bytes), node-wide, outside
--max-memory; a multi-key frame (m,
o, y) is charged 256 bytes per key (288 for
o) before its key lists are built, until its reply is
written; a connection that cannot get its share in time is
closed | — |
| Idle connection timeout | 60 s | 60 s |
A connection over either connection cap is answered B (busy)
and closed. At most 128 such rejections are in that reply at a time, per
process (a short wait for the peer to close, so that the reply is not lost
to a reset); past that a rejected connection is closed at once with a
best-effort reply, which a client that has already sent its first frame sees
as a reset. A source that holds 128 rejected connections open therefore
turns the reply into a reset for the other clients being turned away at
that moment, and for no one else. A TLS listener sends no reply and is not
affected.
The memory bound is approximate (stored key and value bytes, plus 180 bytes per entry, and 750 bytes plus its name's length per non-default namespace — both measured on the container image, so a node full of tiny entries or namespaces sits at about the bound in resident memory); least-recently-used entries are evicted first. How the in-flight budget is charged, and how to size a node process around it, is in Sizing a node process. Use the capacity planner to size nodes, R, and TTL against a target hit rate.