Did the data really change? Detecting real changes with canonical JSON hashing

The problem

The data source service stores the result of each source and refreshes it every night (see "Stop querying live"). Each night, hundreds of results come back. Almost all of them are exactly the same as yesterday.

That raises a simple question with surprisingly deep consequences:

After a refresh, did this source's data actually change?

Knowing the answer matters:

• the sync history can say "177 refreshed, 2 changed" instead of "177 refreshed",
• anything that depends on a source (cached configurators, downstream copies, notifications) only needs to react to the 2 that changed, not reprocess all 177,
• an unexpected change in a source that "never changes" becomes visible.

Comparing two large JSON results directly on every refresh means loading the old one and deep-comparing it. A cleaner way is to store a fingerprint (a hash) next to each result, and compare fingerprints.

The first version of that idea had a subtle bug waiting inside it.

The details

The naive fingerprint

const hash = sha256(JSON.stringify(result));
const changed = hash !== storedHash;

It looks correct. Same data, same string, same hash. Different data, different hash.

Why it would lie

JSON.stringify writes object keys in the order they appear in the object. And that order isn't guaranteed to be stable across runs:

• PostgreSQL's jsonb type doesn't preserve key order. It stores objects in its own internal order. A query that builds JSON can return {"label":"Oak","value":1} one night and {"value":1,"label":"Oak"} another, depending on how the data was produced.
• Endpoint sources (third-party JSON APIs) can return keys in any order between responses.
• Mapping code that builds objects field by field can change insertion order after a refactor.

Those two strings are different, so their hashes are different:

{"label":"Oak","value":1}   → 3f9a…
{"value":1,"label":"Oak"}   → c71e…

But the data is identical. With the naive version, a night where nothing changed could still report changes, and everything downstream would do pointless work, every night, forever. Worse, real changes would drown in fake ones, and the changed flag would become noise people learn to ignore.

Order matters, but not everywhere

The tempting fix is "sort everything". But that would hide real changes:

• Object keys: order has no meaning. {a, b} and {b, a} are the same thing.
• Array elements: order is meaning. A dropdown list ["Oak", "Walnut", "Ash"] reordered to ["Ash", "Oak", "Walnut"] is a real change: users will see the options in a different order.

So the fingerprint must ignore key order and respect array order.

The challenges

• Nested data. Results are arrays of objects with nested objects inside. Key sorting must apply at every level, not just the top.
• Arrays inside objects inside arrays. The normalization must know, at each level, whether it's looking at an object (normalize) or an array (keep as is).
• Determinism across runs and machines. The same data must give the same hash after a restart, a deploy, or on another server.
• Edge cases. null, empty arrays, primitive results and undefined must all hash predictably.

The solution

1. Canonical JSON: sorted keys, untouched arrays

JSON.stringify accepts a replacer function that's called for every value, at every depth. That's the perfect hook:

export function canonicalJson(value: unknown): string {
  return JSON.stringify(value, (_key, raw: unknown) => {
    // Primitives, null and arrays pass through unchanged (array ORDER is kept)
    if (raw === null || typeof raw !== 'object' || Array.isArray(raw)) {
      return raw;
    }
    // Plain objects: rebuild them with keys in sorted order
    const source = raw as Record<string, unknown>;
    return Object.keys(source)
      .sort()
      .reduce<Record<string, unknown>>((sorted, key) => {
        sorted[key] = source[key];
        return sorted;
      }, {});
  });
}

Because the replacer runs recursively, every nested object at every level is rebuilt with sorted keys, while arrays keep their element order. Both versions now produce the same string:

{"label":"Oak","value":1}  ─┐
                             ├─► {"label":"Oak","value":1}
{"value":1,"label":"Oak"}  ─┘

But a reordered list stays different:

[{"label":"Oak"},{"label":"Ash"}]  ≠  [{"label":"Ash"},{"label":"Oak"}]

2. A stable hash on top

import { createHash } from 'crypto';

export function stableHash(value: unknown): string {
  return createHash('sha256')
    .update(canonicalJson(value) ?? 'undefined')
    .digest('hex');
}

• SHA-256 from Node's built-in crypto: no dependency, fast, and collisions are not a practical concern for change detection.
• canonicalJson(undefined) returns undefined (that's how JSON.stringify behaves), so the ?? 'undefined' gives it a fixed input instead of crashing.
• The output is a 64-character hex string, small enough to store on every row.

3. Using it during a refresh

if (outcome.ok) {
  const hash = stableHash(outcome.data);
  await repo.storeResult(record.id, { data: outcome.data, hash, at: new Date() });
  return { ok: true, changed: hash !== record.cachedResultHash };
}

Each refresh result now carries a reliable changed flag, and the sync history records it per source:

{ "code": "SRC-014", "ok": true, "changed": false, "attempts": 1, "durationMs": 182 }
{ "code": "SRC-031", "ok": true, "changed": true,  "attempts": 1, "durationMs": 240 }

A nightly run can now be summarized as "177 refreshed, 2 changed", and anything downstream can be scoped to exactly those 2 codes.

4. The first run is "everything changed", and that's fine

When the hash column is introduced, no row has a stored hash yet. On the first refresh, every source compares against null, so every source reports changed: true. That's correct, not a bug: "we didn't know before, now we do". From the second run on, only real changes show up.

5. Test the properties, not just examples

The tests that matter most check the properties of the function:

expect(stableHash({ a: 1, b: { x: 1, y: 2 } }))
  .toBe(stableHash({ b: { y: 2, x: 1 }, a: 1 }));     // key order ignored, at every depth

expect(stableHash([1, 2, 3]))
  .not.toBe(stableHash([3, 2, 1]));                  // array order respected

expect(stableHash([{ a: 1, b: 2 }]))
  .toBe(stableHash([{ b: 2, a: 1 }]));               // objects inside arrays normalized

Lessons learned

• Same data can serialize differently. Never assume JSON.stringify output is stable when key order isn't under your control, and with jsonb or third-party APIs, it isn't.
• Decide what "equal" means before you hash. Here: key order is noise, array order is meaning.
• A replacer function gives you recursion for free. Normalizing inside JSON.stringify handles every depth with a few lines.
• Fake changes are worse than no detection. A changed flag that's often wrong trains people to ignore it.
• Store fingerprints, compare fingerprints. A 64-character hash per row makes change detection cheap and keeps old data out of memory.
• The first comparison is special. Expect "everything changed" once, and don't treat it as an alert.
• Test properties. "Key order doesn't matter" and "array order does" are the real specification, so they deserve their own tests.

FAQ

Why does PostgreSQL jsonb change the order of keys?

jsonb stores JSON in a binary format optimized for querying, and it doesn't preserve the original key order or duplicate keys. When you read the value back, keys may come out in a different order than you inserted them. The plain json type preserves the text as written.

How do I compare two JSON objects regardless of key order?

Serialize both in a canonical form: sort object keys recursively at every level, keep arrays in their original order, then compare the strings or their hashes.

Should array order be ignored when comparing JSON?

Only if order truly has no meaning in your data. For lists shown to users, ordered steps or rankings, a different order is a real change and should be detected.

Is SHA-256 a good choice for change detection?

Yes. It's built into most platforms, fast enough for typical payloads, and the chance of two different results producing the same hash is negligible. For change detection you don't need a secret or a keyed hash.