One engine, many services: how a configurator backend talks to auth, ERP, folders and data sources

The problem

When people hear "configurator engine", they imagine one application: a database of products and a UI to edit them. In reality, the engine sits in the middle of a small ecosystem:

                    ┌──────────────┐
                    │ Auth service │  identity, sessions, permissions
                    └──────┬───────┘
                           │ JWT · Redis versions · user push
┌──────────┐        ┌──────▼──────────────┐        ┌────────────────┐
│   ERP    │──────▶ │ Configurator engine │ ◀────▶ │ Folder service │
│ (import) │ import └───┬────────┬────────┘  HTTP  └────────────────┘
└──────────┘            │        │
              PostgreSQL│        │HTTP
              (shared)  │   ┌────▼──────────┐
                        │   │ Data sources  │  live lists (materials, prices…)
                    ┌───▼┐  └───────────────┘
                    │ DB │       ┌───────┐
                    └────┘       │ Redis │  locks · presence
                                 └───────┘
        ▲
        │ export (service tokens)
 Other services consuming configurator trees

Every arrow is a place where something can go wrong: a timeout, a missing record, a duplicate, a confusing error, or worse, a cascade, where one slow neighbor makes the whole engine slow.

This post is the map: what each connection does, what can fail, and the rules that keep failures local.

The details

1. The auth service: identity is borrowed, never owned

The engine doesn't store passwords or roles. The central auth service handles login and issues JWTs. The engine:

• verifies tokens locally with the shared secret (fast, no network call),
• reads token and permission versions from the auth service's Redis, so logouts and permission changes take effect instantly (see "Blocking a user mid-session"),
• publishes its endpoint catalog to the auth service: every route declares an action, and the permission names are built by one shared helper, so the permissions admins assign are exactly the ones the guards check,
• receives the user list by push, with versions, to keep a local copy for names and teams (see "Keeping a local copy of users").

Browser sessions use httpOnly cookies (JavaScript never sees the token), and the same token can be read from a Socket.IO handshake for the real-time features.

2. Machine-to-machine: service tokens

Other services consume configurator exports. They have no user, so they use service tokens: recognized by a prefix before parsing, scoped to explicit permissions, with no admin bypass, and revocable through the same version mechanism. The export endpoint also strips UI-only fields (like counters) from its public response, so consumers get a stable contract that isn't tied to the editor's needs.

3. The ERP: importing existing products

Many products already existed in the company's ERP (Odoo) as JSON definitions. Rebuilding them by hand wasn't realistic, so there's an import script that converts them into configurator trees: items, attributes, inputs with min/max/default values, dynamic lists pointing to data sources, and so on.

Imports are where data gets messy, especially duplicates. The script handles them explicitly. When it finds an entity that already exists, it stops and asks:

⚠️  DUPLICATE FOUND: Configurator "Kitchen Cabinet" already exists.
  1) Stop and roll back everything
  2) Skip and use the existing one
  3) Delete and recreate (hard delete, cascade)

"Roll back" has to really mean nothing happened: a half-imported product is worse than no product. But the import is interactive. It can stop and wait for a human answer in the middle, and a database transaction should never stay open for minutes while someone reads a prompt (it would hold locks and a connection the whole time). So the import uses a compensating rollback instead:

const created = { configuratorId: null as number | null, typeIds: [] as number[], actionIds: [] as number[] };

// every create records what it made…
created.configuratorId = (await db.configurator.create({ data })).id;

// …and rollback deletes exactly that, in reverse dependency order
async function rollback() {
  if (created.configuratorId) await db.configurator.delete({ where: { id: created.configuratorId } }).catch(() => {});
  for (const id of created.typeIds)   await db.itemType.delete({ where: { id } }).catch(() => {});
  for (const id of created.actionIds) await db.action.delete({ where: { id } }).catch(() => {});
}

Deleting the configurator cascades to its items and their relations, and shared records created along the way (types, actions) are removed explicitly. Rollback runs when the user chooses option 1 and when the import crashes, so an error never leaves a partial product behind.

4. The folder service: organization lives elsewhere

Configurators are organized in folders (a tree with rename, move, soft delete, restore and trash). That tree is managed by a separate folder service. The engine talks to it through a small dedicated client:

const client = axios.create({
  baseURL: config.get('FOLDER_SERVICE_URL'),
  timeout: 5000, // never wait forever for a neighbor
});

And every error from that service is translated into a meaningful error for the engine's own clients:

private mapError(error: unknown): never {
  if (axios.isAxiosError(error)) {
    const status = error.response?.status;
    const message = error.response?.data?.message;
    if (status === 404) throw new NotFoundException(message ?? 'Folder not found');
    if (status === 409) throw new ConflictException(message ?? 'Folder already exists');
    if (status === 400) throw new BadRequestException(message ?? 'Bad request');
  }
  // timeouts, network errors, 5xx: the neighbor is the problem, not the user
  throw new BadGatewayException('Folder service is unavailable');
}

A user renaming a folder to an existing name gets "Folder already exists" (409), not a generic 500. And when the folder service is down, the answer is 502 Bad Gateway: honest about whose failure it is.

5. Data sources: live lists inside configurators

Some configurator fields don't have fixed options. A "material" list or a price table comes from an external data source, referenced by a key stored as an item attribute. Where the data lives is defined by a source registry in configuration.

When building a tree that needs live data, the engine:

1. walks the tree and collects every source key, deduplicated (ten fields using the "materials" source cause one fetch, not ten),
2. fetches all unique sources in parallel,
3. degrades gracefully per key: if one source fails, that key returns null and the rest of the tree is still delivered.

const results = await Promise.all(
  uniqueKeys.map(async (key) => {
    try {
      const { data } = await http.get(`${registryUrl}${key}`);
      return { key, data };
    } catch (error) {
      logger.error(`Source "${key}" failed`, error);
      return { key, data: null }; // one broken source doesn't break the tree
    }
  }),
);

There's also a batch endpoint to fetch several sources at once, and a background job that pre-builds and caches trees for up to ten configurators at a time, tracked by a sync id the user can follow.

6. Redis: real-time state

Redis holds everything short-lived and shared: edit locks, socket indexes and viewer presence. Two Redis connections exist on purpose: the engine's own (locks, presence) and a read connection to the auth service's Redis (token versions). They have different owners and different lifecycles, and mixing them would blur who is responsible for what.

7. PostgreSQL: shared, so treated with care

The database server is shared with other services. That one fact shaped several decisions: an explicit connection pool cap, no unbounded parallel queries, and bulk loading instead of per-row loops (see "Too many clients" and "Prisma ignored my connection limit").

The challenges

• Every integration fails differently. Timeouts, 404s, duplicates, stale data and out-of-order updates each need their own handling.
• Failures cascade by default. Without timeouts, one slow service ties up the engine's requests, then its connections, then everything.
• Errors lose meaning across boundaries. A 409 from the folder service becomes a useless 500 unless someone maps it.
• Optional data can block essential data. A missing price list shouldn't prevent a configurator from loading.
• Ownership blurs. When two systems both "have" the users, which one is right?

The solution: the rules I follow

Rule 1: every piece of data has exactly one owner

When a question like "is this user's email right?" comes up, the table answers it: ask the owner.

Rule 2: never wait forever

Every outgoing HTTP call has a timeout. A neighbor that doesn't answer in a few seconds is treated as down. Waiting longer only spreads the problem.

Rule 3: translate errors at the boundary

Map remote errors to meaningful local ones (404 → Not Found, 409 → Conflict, timeouts and 5xx → 502 Bad Gateway). Users get actionable messages, and logs show which service failed.

Rule 4: separate essential from optional

The configurator tree is essential; a live price list is optional. Optional data degrades (returns null, logged) instead of failing the whole response.

Rule 5: do once what you'd otherwise do many times

Deduplicate source keys, batch user data, load relations in bulk. Most integration slowness is the same call repeated.

Rule 6: make imports all-or-nothing and explicit

Imports either fully succeed or leave no trace, through a transaction when the import is quick, or a compensating rollback when it's interactive. Every duplicate is a decision, never a silent overwrite.

Rule 7: protect shared infrastructure

On a shared database, cap your pool. On shared Redis, keep separate connections per owner. Your service should never be the one that takes the neighbors down.

Lessons learned

• Draw the map. Listing every connection, what it carries and who owns the data clarified more bugs than any single fix.
• Borrow identity, don't copy it. Keep auth in one place; cache only what you display.
• Timeouts are not optional. An integration without a timeout is a future outage.
• Errors are part of the API contract. A mapped 409 is a feature; an unmapped 500 is a bug report waiting to happen.
• Degrade by importance. Decide in advance which data can be missing and which can't.
• Rollback must be real. "Roll back everything" only works if you know exactly what you created, through a transaction or by tracking every new record.
• Never hold a transaction open while a human thinks. Interactive flows need compensating rollbacks, not long transactions.
• Being a good neighbor is a design goal. Caps, timeouts and bulk queries protect the other services as much as your own.

FAQ

How should microservices handle failures from other services?

Use timeouts on every call, translate remote errors into meaningful local errors, retry only when it's safe, and degrade gracefully for optional data so one failing dependency doesn't break the whole response.

What is a 502 Bad Gateway used for in an API?

It tells the client that your service is working, but a service it depends on returned an invalid response or failed. It's more honest and more useful for debugging than a generic 500.

Should each microservice own its data?

Yes, each piece of data should have one owning service that is the source of truth. Other services may keep read-only copies for performance, with a clear sync and repair strategy.

How do you import data from an ERP safely?

Make it all-or-nothing: use a transaction for short automatic imports, or track every record you create and delete them on failure (a compensating rollback) for long or interactive ones. Detect duplicates explicitly, decide how to handle each one, and validate the converted data before saving.