Manage test data across local runs and CI

Shared state is where flakiness is born

The canonical failure: two tests both operate on test@example.com. Run alone, each passes. Run together, one deletes the user the other is mid-session with. The suite becomes order-dependent — it passes today because of alphabetical file naming, fails Tuesday when someone renames a spec. Retries don’t fix this; they convert a deterministic bug into an intermittent one, which is strictly worse to debug.

The diagnostic is simple: if you can’t run any subset of your suite in parallel against the same environment, your tests share state. The fix is structural, not procedural — no amount of “please clean up after yourselves” survives contact with real parallelism. Everything below exists to make every test hermetic by construction.

Rule 1: unique-per-run data beats cleanup

The cheapest isolation is collision-proof identifiers. Every user, org, project, and invoice a test creates gets a unique suffix generated at runtime:

Example to adapt: helper functions, accounts, and endpoints depend on your test project. Adapt this example to your project.

import { nanoid } from 'nanoid';

export const newUser = (role: 'member' | 'admin' = 'member') => ({
  email: `qa-${runId()}-${nanoid(8)}@example.test`, // .test is reserved, never routable
  plan: 'free',
  role,
});

Note the tradeoff honestly: unique emails make local debugging harder (“which row was mine?”). Mitigate with structure — a stable qa- prefix, the CI run id, then randomness — so a DBA can find everything from one run with a single LIKE 'qa-4812-%'. Never share accounts between tests, even sequentially; “it passed locally” is usually shared-state luck.

Rule 2: deterministic seeds and factories

Randomness inside a test should be seeded or it’s a future flake. If your factory uses Faker, seed it per run (Faker.seed(42)) so failures reproduce. Freeze anything time-shaped: inject a clock rather than reading wall time, so “invoice due next month” means the same thing in August and February.

Prefer factories over committed fixtures. A checked-in users.json drifts silently from your schema until the night someone adds a NOT NULL column. Factories derive data from current code, so schema migrations break loudly at build time instead of softly at 2am. Keep seed scripts versioned alongside migrations — schema and baseline state move as one unit. For deeper background on why order-dependent suites rot, see our flaky-tests causes and fixes.

Rule 3: synthetic identities that announce themselves

Test identities should be unmistakable at a glance and harmless in reality. Email plus-addressing gives you both properties: qa+signup-4812@yourdomain.test routes into one monitored inbox yet stays unique per flow and per run. Adopt a written policy for the rest:

• Names: obviously fake (“QA BOT”, “Test Runner”), never realistic personas.
• Phones: reserved/fictional ranges only, backed by an SMS clone — never a real SIM.
• Emails/SMS delivery: disposable inboxes and cloned providers, not employee mailboxes. We show the OTP pattern for email verification flows and Twilio-style SMS codes.
• Never touch records belonging to real customers, domains, or employees — not even read-only “just to check.”

Marked identities pay off downstream: support can spot them instantly, analytics filters exclude them by prefix, and incident triage never confuses a test storm with an outage.

Rule 4: PII minimization and masking

Don’t copy production data into staging wholesale — it’s the fastest way to turn a test environment into an unmonitored data breach. Prefer generating synthetic data shaped like production (same cardinalities, same edge cases) over masking real people. When masking is unavoidable, do it deterministically: the same input always maps to the same surrogate, so joins, foreign keys, and dedup logic keep working.

Extend hygiene to artifacts: traces, videos, and console logs capture whatever the page rendered. Redact at capture time for anything user-shaped, and treat screenshot storage like production storage, because it effectively is one.

Rule 5: cleanup vs snapshot/restore

Cleanup scripts are the polite fiction of test data management: under parallelism they race. Two runs deleting “their” rows by prefix will eventually collide on a foreign key, a unique index, or a cache. Deletion-based isolation scales poorly precisely when your suite gets fast enough for it to matter.

The structural alternative is snapshot/restore: give each worker an ephemeral database, seed it once from a golden image (schema + baseline data), and let the run trash it freely. Restore is O(seconds), cleanup becomes irrelevant, and parallelism is bounded only by hardware. Costs to weigh: image size, image drift (rebuild the golden snapshot whenever migrations land). The pragmatic hybrid most teams converge on is an ephemeral database per worker, booted with migrations plus a small deterministic seed. Third-party side effects get the same treatment via service clones instead of shared vendor sandboxes — we compare the approaches in API mocking vs service clones, and the wider environment question in staging alternatives.

The six rules, recapped

• Unique-per-run identifiers for everything a test creates — structured, searchable prefixes.
• Determinism everywhere: seeded generators, injected clocks, factories over frozen fixtures.
• Synthetic marked identities: plus-addressed email, fictional phones, clone-backed OTP.
• No raw production PII: synthesize where possible; mask deterministically when not.
• Snapshot/restore over cleanup scripts once the suite runs in parallel.
• Treat artifacts as data: redact traces and screenshots like the production systems they are.

Many apparent flakes are shared-state failures: two tests sharing a row can look like timing bad luck.