Getting started
A single node is one command. A replicated cluster is three more.
Run a single node
Docker
docker pull ghcr.io/nanocached/nanocached-node:latest
docker run --rm --publish 8356:8356 ghcr.io/nanocached/nanocached-node:latest
From source
Requires Rust (the repository pins the toolchain via
rust-toolchain.toml, so Rustup sets everything up):
git clone https://github.com/nanocached/nanocached.git
cd nanocached
cargo run --bin nanocached-node -- --host 0.0.0.0 --port 8356
The node listens on 127.0.0.1:8356 by default; override
with --host and --port. A release build
(cargo build --release) also produces ncd, a
thin dispatcher: ncd node start … /
ncd discovery start ….
Smoke test
printf 'S 4 5\nnameAliceG 4\nname' | nc 127.0.0.1 8356
Expected response: S, then V 5 /
Alice. See the protocol
reference for every command.
Build a cluster
A cluster is N cache nodes plus one or more discovery servers. The discovery server is a lightweight membership registry — it is never on the data path; clients fetch the node list from it and talk to nodes directly (how it works).
Start a discovery server:
docker run --rm --publish 8357:8357 ghcr.io/nanocached/nanocached-discovery:latest
# or: cargo run --bin nanocached-discovery -- --port 8357
Start nodes pointing at it:
cargo run --bin nanocached-node -- --port 8356 --discovery 127.0.0.1:8357
cargo run --bin nanocached-node -- --port 8358 --discovery 127.0.0.1:8357
Each node sends a heartbeat every 5 seconds; the discovery server
derives the node's reachable address from that connection's own source
IP plus the node's port, so containerized deployments need no address
configuration. (Asymmetric port mapping — Docker's
-p 9999:8356 — is unsupported: it is NAT, which the
cluster design already excludes.) New nodes join with a staged data
handoff, so adding a node never drops the keys it takes ownership
of.
Replication factor
Discovery owns the cluster's replication factor R (default 2, minimum 1):
cargo run --bin nanocached-discovery -- --port 8357 --replication-factor 2
Every key lives on its top-R nodes as ranked by rendezvous hashing, so
any single node death costs no cached data. Effective cluster capacity is
total memory ÷ R; --replication-factor 1 restores
single-copy behavior. Clients learn R automatically from the node list —
no SDK configuration.
Discovery replicas (no single point of failure)
The registry is soft state rebuilt from node announces, so discovery replicas run with zero coordination between them. Give every node — and every SDK client — the same list in the same order:
cargo run --bin nanocached-discovery -- --port 8357
cargo run --bin nanocached-discovery -- --port 8358
cargo run --bin nanocached-node -- --port 8356 \
--discovery 127.0.0.1:8357,127.0.0.1:8358
Losing any one replica — including the first-listed primary — costs
neither cache traffic nor client bootstrap; only joins need the
primary up. A restarted replica answers B (busy) for the
liveness-timeout window while live members re-announce, so a
bootstrapping client never sees a half-recovered node list.
Each replica needs its own stable, individually addressable
name (or a stable IP) — an operational requirement the system
does not verify. Every node and client must hold the same list in the
same order; a single DNS name that round-robins across several
replicas (or a multi-value discovery service, e.g. AWS Cloud Map's
multi-value routing) breaks that, since which replica a resolution lands
on — and in what order — becomes nondeterministic per resolver, per
lookup. A bare task/pod IP has the same problem in a different shape: it
works until that replica restarts with a new one, silently dropping it
from every --discovery list. Give each replica a stable
identity instead — on ECS, one Cloud Map service (or ALB target group)
per replica rather than one round-robin service in front of all of them;
on Kubernetes, a headless Service-per-replica or a StatefulSet's stable
per-pod DNS names.
Every replica must be started with the same
--replication-factor — nothing enforces this at startup, since
replicas never talk to each other. A node reports the replication
factor it has learned on every heartbeat; while any joined node's
reported value disagrees with a replica's own configured value, that
replica logs a loud WARN and refuses node-list requests
with B (the same "retry elsewhere" answer as its startup
grace) rather than serving a replica count known to disagree with what
part of the cluster learned elsewhere. Clients simply skip to the next
seed, so a partially rolled-out misconfiguration degrades loudly
instead of silently splitting routing.
Authentication
Set NANOCACHED_AUTH_SECRET to require clients to
authenticate before any other command. It is an environment variable —
not a flag — so the secret never shows in ps output. Set it
on both roles to protect a cluster:
NANOCACHED_AUTH_SECRET=change-me nanocached-node --port 8356 --discovery 127.0.0.1:8357
NANOCACHED_AUTH_SECRET=change-me nanocached-discovery --port 8357
Nodes reuse the same value to authenticate their own heartbeats.
Unset (or empty) disables authentication entirely. Authentication is a
second layer of defense, not a substitute for network isolation: without
TLS there is no transport encryption, so bind to 127.0.0.1
or a private interface and add TLS when the network is not trusted.
TLS
Pass --tls-cert/--tls-key (PEM) to require
TLS on every accepted connection — no plaintext fallback once set:
nanocached-node --port 8356 --tls-cert cert.pem --tls-key key.pem
nanocached-discovery --port 8357 --tls-cert cert.pem --tls-key key.pem
A node registering with a TLS-secured discovery server also needs
--tls-ca (the CA bundle to trust for its heartbeat
connection — only those CAs, not the system store, since cluster
certificates are typically private):
nanocached-node --port 8356 --tls-cert cert.pem --tls-key key.pem \
--tls-ca ca.pem --discovery 127.0.0.1:8357
For local development, generate a self-signed certificate that rustls
will accept as a leaf (CA:FALSE matters — OpenSSL's default
self-signed certs are marked as their own CA and get refused):
openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 1 \
-subj "/CN=127.0.0.1" \
-addext "subjectAltName=IP:127.0.0.1" \
-addext "basicConstraints=critical,CA:FALSE" \
-addext "keyUsage=critical,digitalSignature,keyEncipherment" \
-addext "extendedKeyUsage=serverAuth"
Being self-signed, the same cert.pem doubles as the
--tls-ca trust anchor for anything connecting to it.
Connect a client
Use any of the six SDKs — each detects whether it reached a node or a discovery server from the handshake itself, so single-node development and clustered production use identical code. See SDKs.
Built-in limits
| Limit | nanocached-node | nanocached-discovery |
|---|---|---|
| Maximum request size | 1 MiB | 4 KiB |
| Maximum concurrent connections | 1,024 by default
(--max-connections; per source IP 256 by default,
--max-connections-per-ip — the effective ceiling behind
NAT or on Kubernetes, where clients share a source IP) | 1,024 (fixed) |
| Cache memory bound | 256 MiB by default (--max-memory), LRU eviction;
per-namespace caps via --namespace-budget <ns>=<bytes>
(a namespace over its cap evicts from itself first) | — |
| Idle connection timeout | 60 s | 60 s |
The memory bound is approximate (sum of stored key and value bytes); least-recently-used entries are evicted first. Use the capacity planner to size nodes, R, and TTL against a target hit rate.