Meridian.

The problem

A live-service game needs fast access to session state, player ratings, and leaderboards. Meridian is my from-scratch exploration of that backend: a C++20 in-memory cache server with a text protocol, concurrent cache access, and explicit control over the networking layer.

The interesting part is the interaction between small decisions. A cache eviction policy changes the locking requirements. A sharding strategy changes which keys move. A write-ahead log becomes the basis for replication.

Follow a request

Non-blocking sockets are driven by kqueue on macOS and epoll on Linux. The server handles commands split across packets, pipelined commands, backpressure, and TCP half-close. The event-loop interface keeps readiness handling separate from the protocol.

The cache path routes a key through a consistent hash ring with 128 virtual nodes per shard. Within the selected shard, the key chooses a cache stripe. Leaderboard commands take a separate path to a skip list owned by the event-loop thread.

Why an LRU read needs a lock

The LRU cache combines a hash map with an intrusive doubly linked list. The map finds an entry; the list tracks recency. A successful get moves an entry toward the most-recently-used end. A read changes the data structure.

That makes reader-writer locking less useful here: cache hits still mutate shared state. Meridian partitions the keyspace into independent LRU caches, each protected by its own mutex.

A hash detail that matters

Stripe selection remixes the key hash before taking the remainder. The inner unordered_map also buckets by hash; reusing the same modular pattern could concentrate a stripe’s keys into too few map buckets.

// src/cache/striped_cache.cpp
return mix64(std::hash<std::string>{}(key))
       % stripes_.size();

TTL expiration is lazy: entries are checked on access. The clock is injectable, allowing expiry tests to advance time deterministically instead of sleeping.

Measure the cache, specifically

The repository reports 838K → 10.3M operations per second when moving from one global lock to 64 stripes. All three runs use an Apple M3, eight threads, four million operations, and an in-process hot-key workload. The reported hit rate is 92.39% for each configuration.

To reproduce the workload after a Release build, vary N:

./build-release/load_gen --threads 8 --stripes N

Other decisions worth opening

Consistent hashing makes movement predictable

When a shard leaves the ring, keys that belonged to surviving shards keep their assignments. The hash-ring tests assert that invariant directly. The wire command SHARD <key> exposes the routing decision.

Rank queries need more than a sorted list

The leaderboard uses a hand-built skip list with span-annotated links. Spans make rank queries possible without walking every lower-ranked entry. The repository describes a randomized comparison against a brute-force reference model.

One mutation log serves recovery and replication

Successful mutations are appended as readable protocol lines. Recovery replays them through the same command handler. A follower requests the backlog, then receives the live mutation stream and rejects client writes. A torn final log line is detected and truncated on reopen.

This is leader/follower replication without consensus or automatic failover. HTTP/JSON APIs, durable TTLs, and snapshots are still listed as planned work in the repository’s v1 roadmap.

Inspect the implementation

• README.md ↗
  Architecture, benchmark conditions, build instructions, and current status.
• src/cache/striped_cache.cpp ↗
  Per-stripe locks and hash remixing.
• src/cache/lru_cache.cpp ↗
  Recency updates, eviction, and lazy expiration.
• src/net/tcp_server.cpp ↗
  Socket handling and protocol buffering.
• src/cluster/hash_ring.cpp ↗
  Virtual nodes and ring lookup.
• src/replication/wal.cpp ↗
  Readable mutation logging and recovery.
• ROADMAP.md ↗
  The implemented baseline and remaining v1 scope.