Ir al contenido

Upgrading your in-memory store from Redis to Valkey

Administrar un servidor 9 min de lectura Fuente en codefloeSolo en inglés

En esta página

Catodon now ships Valkey instead of Redis as its bundled in-memory store (used for caching, pub/sub, and the job queue). This page is a step-by-step runbook for moving an existing installation over. For the reasoning behind each step, see the longer reference in Installing and running Catodon — this page exists to be followed, not to be re-derived from first principles.

The application itself needs no changes. Valkey is a fork of Redis 7.2 and speaks the same protocol. The redis / redisFor* keys in .config/default.yml keep their names, ioredis stays the client, and pointing Catodon at an existing Redis server continues to work. Only the store you run changes.

Why Valkey, and how far Redis is still supported. Valkey is the store Catodon ships, tests and documents. Redis still works — it speaks the same protocol and the config keys are unchanged — but it is a compatibility path rather than a second supported option: Catodon does not test it, the two projects are already diverging, and a future Redis release may stop working here. The licenses point the same way: Catodon is AGPLv3 and Valkey is BSD-3, a fork of the last BSD-licensed Redis (7.2), while Redis 7.4 and later are under RSALv2/SSPLv1.

Pick your path:

Both paths share the same three ideas, explained once here so the steps below can stay short:

  1. Check your Redis version before touching anything. Valkey can only reuse a data file written by Redis 2.x–7.2.x. redis:7-alpine (the tag Catodon used to ship) floated from 7.2 to 7.4 over time, so the tag alone doesn't tell you — you have to ask the running server.
  2. Drain the queue before you stop the old store. Pending federation deliveries and (if you enabled it) buffered reaction counts live only in Redis/Valkey. Postgres cannot rebuild them. This has to happen while the old store and Catodon are both still running. Note what draining does not cover: notification history has no queue to flush and no table to fall back on, so the only thing that keeps it is reusing the data file.
  3. Keep a backup, and keep the old store around until you've verified the new one. The move is one-way in practice: once Valkey 9 has written its own dump, going back to Redis (or Valkey 8) is not reliable.

Step 0: Check your Redis version

Run this against your current, running store, before changing any config or compose file:

# Docker:
docker compose exec redis redis-cli INFO server | grep redis_version

# Bare metal:
redis-cli INFO server | grep redis_version

Note the result — you'll need it in step 2.

  • 7.2.x or older → your data file is compatible with Valkey. You'll reuse it.
  • 7.4 or newer → your data file is not compatible with Valkey. You'll start Valkey with an empty data directory instead (see "If you're on Redis 7.4+" below).

Step 1: Drain the job queue (required)

This must happen before you stop the old store. The admin dashboard and workers disappear the moment Catodon stops, so there's no doing this later.

  1. Open Control Panel → Server health.
  2. Watch the deliver queue until wait, active, delayed, and prioritized are all at (or near) zero. Check all four — wait alone can read empty while failed deliveries are still sitting as delayed retries, sometimes for hours.
  3. If you use reactions buffering (off by default — skip this if you never turned it on): switch it off in the control panel. This queues a bakeBufferedReactions job on the system queue. Watch that queue the same way until the job is gone from wait/active/delayed/prioritized. A failed job means the deltas are still owed — check the error and retry; it's safe to retry.
  4. Optional CLI cross-check: the queue keys and buffer keys are prefixed with your instance hostname when redis.prefix is unset, e.g. <your.host>:queue:deliver:deliver:wait and <your.host>:reactionsBufferDeltas:*. Querying without that prefix returns 0 and looks empty even when it isn't.

If delayed is the only count still above zero, you don't have to wait out the backoff. Open that queue and press Promote all jobs (or call admin/queue/promote-jobs): it moves the queue's delayed jobs back to wait so the workers retry them now. It acts on whichever queue is open, and a job that failed because a remote server was unreachable will most likely fail again and return to delayed, so this shortens the wait rather than guaranteeing the drain.

Once both queues are clear, you're safe to stop the stack.

Step 2: Back up the old data directory

Still on the old store, now that it's drained:

# Docker:
docker compose down
cp -a redis redis.backup

# Bare metal (path depends on your distro, commonly /var/lib/redis):
sudo systemctl stop redis-server   # or: redis, redis-server, valkey — check your unit name
sudo cp -a /var/lib/redis /var/lib/redis.backup

Keep this backup until you've verified the new store is working. Don't delete it as a cleanup step days later without checking first.

Step 3: Install/start Valkey

Docker

Edit your compose file:

  • Rename the redis service to valkey.
  • Change the image to valkey/valkey:9.1.2-alpine (or newer).
  • Change the healthcheck to valkey-cli ping (the Valkey image doesn't ship redis-cli).
  • Add a redis network alias so .config/default.yml (which still says host: redis) keeps resolving — no need to edit it:
  valkey:
    restart: always
    image: valkey/valkey:9.1.2-alpine
    networks:
      meow:
        aliases:
          - redis
    volumes:
      - ./valkey:/data
    healthcheck:
      test: "valkey-cli ping"
      interval: 5s
      retries: 20

Then, depending on the version you found in step 0:

Redis 7.2.x or older — reuse the directory:

mv redis valkey

Redis 7.4 or newer — start empty (see "Starting empty is lossy" below):

mkdir valkey

Bring the stack up:

docker compose up -d

Bare metal / system packages

Install Valkey from your distribution's package manager, e.g.:

# Debian/Ubuntu
sudo apt install valkey

# Fedora
sudo dnf install valkey

# Arch
sudo pacman -S valkey

If your distribution doesn't package Valkey yet, you can stay on Redis — Catodon works with either — or install Valkey from the upstream instructions.

Redis 7.2.x or older — reuse the data file. Valkey and Redis use the same default data directory conventions in most distro packaging (commonly /var/lib/redis), but confirm the path in both your old and new service's config (dir in redis.conf / valkey.conf) before moving anything:

# Stop the old service first (you already did this in step 2)
sudo cp -a /var/lib/redis/dump.rdb /var/lib/valkey/dump.rdb
sudo chown valkey:valkey /var/lib/valkey/dump.rdb
sudo systemctl start valkey

Redis 7.4 or newer — start Valkey with a fresh/empty data directory instead (see "Starting empty is lossy" below):

sudo systemctl start valkey

Once Valkey is up and responding to valkey-cli ping, point Catodon at it. If Valkey listens on the same host/port as Redis did (localhost:6379 by default), .config/default.yml's redis: block needs no changes at all. Restart Catodon:

pnpm run migrate   # only needed if you also upgraded Catodon itself
pm2 restart catodon   # or however you manage the process

Step 4: Verify, then retire the old store

  • Check Control Panel → Server health: jobs should be processing normally.
  • If you use reactions buffering, turn it back on and confirm new deltas are being counted.
  • Check admin/server-info (or the queue dashboard) reports a valkey_version.

Only once you're satisfied it's stable, remove the old Redis service/package and delete the backup. Until then, keep both around — if something's wrong with the Valkey store, the fastest recovery is switching back to the untouched Redis backup, not debugging forward.


If you're on Redis 7.4+: starting empty is lossy, not free

Much of what's in the store is disposable cache and does rebuild on its own. The main fanout timelines fall back to a Postgres query when their list comes back empty, as long as enableFanoutTimelineDbFallback is on, which is the default, and rate-limit counters simply start counting again.

Four things do not come back. This is why step 1, draining the queue, is required rather than optional, and also why it is not enough on its own: it covers the deliveries and the reaction counts, and can do nothing about the other two.

  • Every account's notifications. There is no notification table. MiNotification is a plain type, and NotificationService appends each notification to a stream per user, notificationTimeline:<userId>, with the read state beside it in latestReadNotification:<userId> and readNonDirectMessageNotifications:<userId>. Discarding the store empties the notification list of every account on the instance, permanently, and nothing re-derives it. Draining does not help, because there is nowhere to drain it to. Watch out for the symptom that reads as "notifications are fine": the Mentions tab keeps working, since notes/mentions queries Postgres and never touches the store.
  • Pending federation deliveries. Postgres holds your notes, but it doesn't hold a record of "this activity still needs to be delivered to these remote servers", which only lives in the deliver queue. Anything still queued when you discard the store is silently never delivered.
  • Buffered reaction counts, only if you turned reactions buffering on. Deltas accumulate in the store and are normally written to Postgres by a daily bake job; discarding the store loses everything since the last bake and leaves those reaction counts permanently low.
  • Custom feeds timelines. antennas/notes reads antennaTimeline:<antennaId> and answers an empty page when that list is empty; unlike the timelines above, it has no database fallback. Custom feeds come back blank and refill only as new matching posts arrive, so this one does heal going forward, but the back catalogue does not return. That list also holds only the most recent ~200 matching posts, dropping the oldest as new ones arrive, so it was never a complete archive; if you rely on a custom feed to keep posts, bookmark them instead.

Two of these you can avoid by draining. Two you cannot, and draining does nothing for them. On Redis 7.4+, migrating means the notification history goes, because the data file cannot be reused and no other step here carries it over; the only path this runbook describes that keeps it is not to migrate, as the next section explains. That is the real price of the 7.4+ path, and it is worth weighing before you start rather than discovering afterwards.

If you'd rather stay on Redis

Staying works for now: the application doesn't care which server answers, and the redis / redisFor* keys are unchanged. But it is a compatibility path, not a second supported one — Catodon doesn't test Redis, the two are diverging, and a future Redis release may stop working, as above. If you stay, just don't take the Valkey image/healthcheck without also switching the data directory logic above. In Docker, this means leaving the service as-is (or, if you already renamed it, keeping the old image and healthcheck together):

  valkey:            # name doesn't matter, only the alias below does
    image: redis:7-alpine
    healthcheck:
      test: "redis-cli ping"

Downgrading back to Redis or to Sharkey

Valkey 8+ changed its dump file's magic header, so an older Redis cannot read a Valkey-written dump at all — it fails cleanly at startup rather than loading corrupt data. If you need to go back:

  • Restore from the pre-migration backup (step 2), not a dump Valkey itself produced.
  • If you're downgrading Catodon back to Sharkey specifically, see Downgrading a Catodon instance to Sharkey for the full procedure, including moving the Valkey directory aside so Sharkey's Redis doesn't refuse to start.