Graph Data Modeling

Migrating to NODE KEY constraints without downtime

You started with a single-property uniqueness constraint — account_id IS UNIQUE — and the model has outgrown it. Identity is now a tuple: an account is unique within a tenant, so the real key is (tenant_id, account_id), and every part of it must always be present. The correct Neo4j 5.x guarantee for that is a composite NODE KEY, but the database is live, taking reads and writes, and you cannot take it offline to swap constraints. This task walks the exact sequence that gets you there with zero downtime: clean the data the new key demands, add the NODE KEY alongside the old constraint, wait for its backing index to come online, and only then drop the rule you are replacing. It is one focused move within the broader constraint lifecycle discipline.

Prerequisites

Why a NODE KEY cannot be swapped in place

Neo4j 5.x has no ALTER CONSTRAINT. You cannot promote account_id IS UNIQUE to (tenant_id, account_id) IS NODE KEY with a single statement, because the two rules own different backing indexes — a single-property RANGE index versus a composite one. The migration is therefore additive: the new constraint must be created and brought online while the old one still guards writes, so there is never a moment when identity is unprotected. And a NODE KEY carries two guarantees at once — joint uniqueness and existence of every member — so it will refuse to be created if any node has a null tenant_id or if two nodes already collide on the composite tuple. The data must satisfy the new invariant before the DDL, not after.

Core implementation

The whole migration is four ordered phases in one script. The comments mark why each phase must precede the next; reordering them reintroduces the downtime or the race you are trying to avoid.

python
from neo4j import GraphDatabase

# PHASE 1 — BACKFILL the new key property so no member is null. A NODE KEY
# requires every property in the tuple to exist; a single null makes the
# CREATE abort. Backfill deterministically from a stable source, in batches.
BACKFILL = """
MATCH (a:Account) WHERE a.tenant_id IS NULL
CALL (a) {
  SET a.tenant_id = coalesce(a.legacy_tenant, 'UNASSIGNED')
} IN TRANSACTIONS OF 10000 ROWS
"""

# PHASE 2 — DEDUP any composite-key collisions. Two rows that were distinct
# under account_id alone may collide once tenant_id joins the key only if the
# data is dirty; surface and resolve them BEFORE the constraint DDL.
FIND_COLLISIONS = """
MATCH (a:Account)
WITH a.tenant_id AS t, a.account_id AS id, collect(a) AS rows, count(*) AS n
WHERE n > 1
RETURN t, id, n ORDER BY n DESC
"""

# PHASE 3 — ADD the composite NODE KEY. Idempotent, and it coexists with the
# old single-key constraint. Its backing composite RANGE index provisions
# asynchronously, so this returns before enforcement is fully live.
ADD_NODE_KEY = """
CREATE CONSTRAINT account_tenant_key IF NOT EXISTS
FOR (a:Account) REQUIRE (a.tenant_id, a.account_id) IS NODE KEY
"""

# PHASE 4 — DROP the superseded single-property constraint. Only after the new
# key is ONLINE. DROP CONSTRAINT removes the rule AND its owned index together.
DROP_OLD = "DROP CONSTRAINT account_id_unique IF EXISTS"


def migrate(uri: str, auth: tuple[str, str]) -> None:
    with GraphDatabase.driver(uri, auth=auth) as driver:
        driver.verify_connectivity()
        with driver.session(database="neo4j") as session:
            # 1. Backfill nulls out of existence.
            session.run(BACKFILL).consume()

            # 2. Refuse to proceed while collisions remain — fail loud, not silent.
            collisions = session.run(FIND_COLLISIONS).data()
            if collisions:
                raise SystemExit(f"Resolve {len(collisions)} key collisions first")

            # 3. Add the new NODE KEY, then BLOCK until its index is ONLINE.
            session.run(ADD_NODE_KEY).consume()
            session.run("CALL db.awaitIndexes(600)").consume()

            # 4. Old constraint is now redundant — retire it and its index.
            session.run(DROP_OLD).consume()

Throughout phases 1 to 3 the original account_id_unique constraint is still enforcing, so writes are never unprotected — that overlap is what makes the migration zero-downtime. The db.awaitIndexes gate between phases 3 and 4 is the load-bearing line: drop the old constraint before the new one is online and there is a window where neither is enforcing composite identity.

Constraint state before and after the NODE KEY migration Left: a Before state where an Account node is protected by a single-property uniqueness constraint on account_id, tenant_id may be null. Center: four stacked ordered phases — backfill tenant_id, dedup collisions, add NODE KEY IF NOT EXISTS, await ONLINE then drop old. Right: an After state where the Account node is protected by a composite NODE KEY on (tenant_id, account_id), both mandatory and jointly unique. An arrow runs left to right through the phases. Before :Account account_id ✓ tenant_id? UNIQUENESS account_id only 1 · backfill tenant_id 2 · dedup collisions 3 · add NODE KEY 4 · await ONLINE, drop old old constraint still enforces through phase 3 → zero downtime After :Account tenant_id ✓ account_id ✓ NODE KEY composite · mandatory

Validation & verification

Prove three things: the new key is live and index-backed, the old one is gone, and no data slipped past the tuple invariant. First inspect the constraint set — the NODE KEY should own a composite index and the old uniqueness row should be absent:

cypher
SHOW CONSTRAINTS YIELD name, type, labelsOrTypes, properties, ownedIndex
WHERE 'Account' IN labelsOrTypes;

Then confirm the backing index reached ONLINE rather than lingering in POPULATING, which would mean writes are still scanning:

cypher
SHOW INDEXES YIELD name, type, state, owningConstraint
WHERE owningConstraint = 'account_tenant_key'
RETURN name, type, state;   // expect type RANGE, state ONLINE

Finally, run a behavioural check: no composite duplicates and no null members should remain, and a fresh insert with a null tenant_id should now be rejected. A zero-row result from the first query is the proof the key holds:

cypher
MATCH (a:Account)
WITH a.tenant_id AS t, a.account_id AS id, count(*) AS n
WHERE n > 1 OR t IS NULL
RETURN t, id, n;   // must return zero rows

Edge cases & gotchas

1. Dropping the old constraint too early. If you issue DROP CONSTRAINT account_id_unique before db.awaitIndexes confirms the NODE KEY is online, there is a window where composite identity is enforced by neither rule and a concurrent writer can insert a colliding tuple. Always await the new index first, as the script does between phases 3 and 4.

2. ConstraintValidationFailed on dirty data. If a null tenant_id or a composite collision survives your backfill and dedup, the CREATE CONSTRAINT aborts and no constraint is created. Do not catch and ignore it — re-run the collision query, resolve the offenders, and retry:

cypher
// Re-surface exactly what blocked the NODE KEY so it can be quarantined.
MATCH (a:Account) WHERE a.tenant_id IS NULL RETURN a.account_id LIMIT 50;

3. Live writes racing the backfill. If the application keeps inserting rows with null tenant_id during phase 1, the backfill never fully drains. Either ship the application change that always populates tenant_id first, or run the backfill and the CREATE CONSTRAINT close together so the new NODE KEY starts rejecting nulls as soon as it is online.

This task is one operation within Constraint Lifecycle Management; the same add-await-drop ordering applies to any constraint replacement, not just NODE KEY promotion.

Related pages