Making an admin change apply everywhere, immediately: the writer side of Redis versions

The problem

In our central auth service, each app's access token carries the user's permissions for that app, and stays valid for 8 hours. That's what makes requests fast: apps verify tokens locally, without calling the auth service.

But it creates an obvious problem. An admin removes "delete invoices" from the Finance group at 9:00. Without anything else, every Finance member keeps that permission until their token expires, possibly at 17:00.

Connected apps already know how to check a version number on every request (that side is described in "Blocking a user mid-session"). This post is about the other side, the harder one: when to change those numbers, and for whom.

Get it wrong in one direction, and changes silently don't apply to some users. Get it wrong in the other direction, and a routine permission edit logs dozens of people out of apps that had nothing to do with it.

The details

Two counters, two meanings

Redis holds two kinds of version numbers:

user:{userId}:tokenVersion                       ← one per user, global
user:{userId}:app:{appId}:permissionsVersion     ← one per user AND per app

Every token is minted with a copy of both. Apps compare them on every request.

• tokenVersion means "is this session still alive?". A mismatch means log in again.
• permissionsVersion means "are this token's permissions still current?". A mismatch means quietly get a fresh token and continue.

They're bumped for completely different reasons:

Mixing them up is the classic mistake. If a permission tick bumped tokenVersion, every affected user would be logged out of every app. A routine admin click would behave like a security incident.

Why permissionsVersion is per (user, app)

A group is global, but its permissions are per app. If the permissions version were one number per user, changing Finance's billing permissions would make Finance members' inventory tokens look stale too. Every app would refresh for nothing.

Keying by (user, app) means a change in billing only touches billing tokens. Inventory sessions don't even notice.

The challenges

Every admin action has its own blast radius

There are many admin actions, and each one affects a different set of users and apps. Each needs its own, exact answer to "whose (user, app) pairs changed?".

Root Admins aren't in any group

Root Admins bypass groups entirely. Any rule that computes affected users "through groups" silently misses them, with no error at all.

Some answers disappear on delete

When a group or a permission is deleted, cascades remove the rows that say who was affected. If you compute the list afterwards, it's empty.

A bump alone doesn't remove access

A stale token makes the app ask for a fresh one. If the auth service then happily mints a new token, nothing changed. The bump only works if the minting paths re-check everything.

Only one writer

Several services read these counters. If more than one service could write them, nobody could reason about who invalidated what.

The solution

1. A table for every admin action

The core of the design is a table, written down before writing code:

Each row was reasoned out case by case:

• Assign uses the union of old and new apps: the user loses rights in some apps and gains them in others. Apps in neither set keep their valid tokens.
• Rename bumps nothing: no right changed, so there's no reason to refresh anyone's token.
• App events add Root Admins explicitly, because a group-based list would miss them.

2. One helper, many (user, app) pairs

All bumps go through one method that takes users × apps:

async bumpPermissionsForUsersInApps(userIds: number[], appIds: number[]) {
  if (userIds.length === 0 || appIds.length === 0) return;
  const pipeline = redis.pipeline();
  for (const userId of userIds) {
    for (const appId of appIds) {
      pipeline.incr(`user:${userId}:app:${appId}:permissionsVersion`);
    }
  }
  await pipeline.exec(); // one round trip, however many pairs
}

INCR is atomic in Redis, so concurrent bumps never lose an increment. A pipeline sends every increment in one round trip, so a department of 40 people across 3 apps is 120 increments in one network call.

3. Read before you delete

Deleting a group is the clearest example:

async deleteGroup(groupId: number) {
  // Both lists must be read BEFORE the delete: the cascade removes the
  // activation rows, and members' groupId is cleared by the foreign key.
  const memberIds = await groups.findMemberIds(groupId);
  const appIds = await groups.findAppIds(groupId);

  await groups.delete(groupId);

  // Former members lose access everywhere this group reached
  await versions.bumpPermissionsForUsersInApps(memberIds, appIds);
}

The same rule applies to pruning permissions from the catalog and to hard-deleting an app: resolve the affected set first, then delete.

4. Don't forget the Root Admins

async bumpForApp(appId: number) {
  const [memberIds, rootAdminIds] = await Promise.all([
    groups.findMemberIdsByApp(appId),
    // Root Admins skip the group check everywhere a token is minted,
    // so a group-based list would silently miss them.
    users.findActiveRootAdminIds(),
  ]);
  const userIds = [...new Set([...memberIds, ...rootAdminIds])];
  await this.bumpPermissionsForUsersInApps(userIds, [appId]);
}

This one bug class is worth remembering: special users who bypass a rule are also invisible to any code that relies on that rule.

5. Bumping is half of it: minting must re-check

A bump makes the token stale. The app then asks the auth service for a new token, through a session exchange or a refresh. That request re-runs every check:

• is the session or refresh token still valid (tokenVersion)?
• is the user still active?
• is the app still active? (checked before the Root Admin shortcut)
• does the user still have access to this app?

If access was removed (not just changed), the auth service refuses with a 403 instead of minting a new token. That's what actually ends the user's session in that app. Without the re-checks, the bump would only hand out a fresh, still-valid token.

6. Keep a permission cache in Redis, and let it heal itself

Besides versions, the auth service also caches each user's resolved permissions per app in Redis (user:{id}:app:{appId}:permissions). Every login, session exchange and refresh re-writes that cache with the permissions it just resolved:

await versions.cachePermissions(userId, appId, resolved.permissions);

So the cache heals itself: any user who logs in after the cache was introduced gets a correct entry without a backfill job. Connected apps can read the current permissions without decoding a token or calling the auth service.

The bump helper also uses the cache, and the order is deliberate: it first re-resolves and re-writes the permission lists for every affected (user, app) pair, and only then increments the versions:

async bumpPermissionsForUsersInApps(userIds: number[], appIds: number[]) {
  if (userIds.length === 0 || appIds.length === 0) return;

  // 1. Refresh the cached lists first...
  await this.refreshPermissionsForUsersInApps(userIds, appIds);

  // 2. ...then bump the versions
  const pipeline = redis.pipeline();
  for (const userId of userIds) for (const appId of appIds) {
    pipeline.incr(`user:${userId}:app:${appId}:permissionsVersion`);
  }
  await pipeline.exec();
}

If the versions were bumped first, an app could read the new version together with the old list for a split second, and cache the wrong thing as "current". Writing the data before announcing the change means no reader can ever see a new version paired with a stale list.

7. The auth service is the only writer

The rule, written into the architecture:

The auth service is the only service allowed to write authentication and authorization state into Redis. App backends only read it.

Apps never increment, reset or delete these keys. That keeps the auth service the single source of truth, and makes every invalidation traceable to one admin action in one service. Each bump is logged with what it touched:

Assigned user 15 to group 4 (was 2); refreshed 3 app(s)
Deleted group 7; unassigned 12 user(s) across 2 app(s)
Refreshed permissionsVersion for 41 user(s) in application 3

What the user experiences

1. 09:00. An admin unticks "delete invoices" for Finance in billing.
2. The auth service bumps permissionsVersion for every Finance member, for billing only.
3. A Finance user clicks something in billing at 09:01. The app sees the version mismatch and answers "stale permissions".
4. The frontend refreshes the token silently. The user's cached permission list for billing (already updated before the bump) no longer contains "delete invoices".
5. The user continues working. The delete button is gone. Their inventory session never noticed anything.

Lessons learned

• Separate "session dead" from "permissions changed". One logs people out; the other quietly refreshes. Never use the first for routine admin work.
• Scope versions to what changes together. Per (user, app) means a billing change never disturbs inventory.
• Write the blast-radius table first. Every admin action deserves an explicit answer to "which users, which apps?".
• Watch for users who bypass your rules. Root Admins skip groups, so group-based logic will miss them.
• Read before you cascade. Collect affected users and apps before deletes erase them.
• Invalidation needs enforcement. A stale token is only useful if minting re-checks access and can refuse.
• One writer, many readers. Shared state stays understandable when only one service can change it.
• Let caches heal on the normal path. Re-writing the cache on every login removes the need for backfill jobs.

FAQ

How do I make permission changes apply immediately with JWTs?

Store a version number per user (and per app if permissions are per app) in a shared store like Redis, embed it in each token, and compare on every request. Increment it when permissions change, so old tokens are detected as stale and refreshed.

Should a permission change log the user out?

Usually not. A permission change can be handled by refreshing the token in the background. Logging out should be reserved for security events like disabled accounts or forced logouts.

Why key permission versions by user and application?

Because a change in one application shouldn't invalidate tokens for other applications. Per-app versions limit refreshes to the app where permissions actually changed.

Is Redis INCR safe under concurrency?

Yes. INCR is atomic, so concurrent increments are never lost. Pipelining many INCR commands sends them in one round trip without changing that guarantee.