database-migrations

Schema Evolution Strategies

StrategyRiskDowntimeBest ForAdditive-OnlyVery LowNoneAPIs with backward-compatibility guaranteesExpand-ContractLowNoneRenaming, restructuring, type changesParallel ChangeLowNoneHigh-risk changes on critical tablesLazy MigrationMediumNoneLarge tables where bulk migration is too slowBig BangHighYesDev/staging or small datasets only Default to Additive-Only. Escalate to Expand-Contract only when you must modify or remove existing structures.

Zero-Downtime Patterns

Every production migration must avoid locking tables or breaking running application code. OperationPatternKey ConstraintAdd columnNullable firstNever add NOT NULL without default on large tablesRename columnExpand-contractAdd new → dual-write → backfill → switch reads → drop oldDrop columnDeprecate firstStop reading → stop writing → deploy → dropChange typeParallel columnAdd new type → dual-write + cast → switch → drop oldAdd indexConcurrentCREATE INDEX CONCURRENTLY — don't wrap in transactionSplit tableExtract + FKCreate new → backfill → add FK → update queries → drop old columnsChange constraintTwo-phaseAdd NOT VALID → VALIDATE CONSTRAINT separatelyAdd enum valueAppend onlyNever remove or rename existing values

Migration Tools

ToolEcosystemStyleKey StrengthPrisma MigrateTypeScript/NodeDeclarative (schema diff)ORM integration, shadow DBKnexJavaScript/NodeImperative (up/down)Lightweight, flexibleDrizzle KitTypeScript/NodeDeclarative (schema diff)Type-safe, SQL-likeAlembicPythonImperative (upgrade/downgrade)Granular control, autogenerateDjango MigrationsPython/DjangoDeclarative (model diff)Auto-detectionFlywayJVM / CLISQL file versioningSimple, wide DB supportgolang-migrateGo / CLISQL (up/down files)Minimal, embeddableAtlasGo / CLIDeclarative (HCL/SQL diff)Schema-as-code, linting, CI Match the tool to your ORM and deployment pipeline. Prefer declarative for simple schemas, imperative for fine-grained data manipulation.

Rollback Strategies

ApproachWhen to UseReversible (up + down)Schema-only changes, early-stage productsForward-only (corrective migration)Data-destructive changes, production at scaleHybridReversible for schema, forward-only for data

Data Preservation

Soft-delete columns — rename with _deprecated suffix instead of dropping Snapshot tables — CREATE TABLE _backup_<table>_<date> AS SELECT * FROM <table> Point-in-time recovery — ensure WAL archiving covers migration windows Logical backups — pg_dump of affected tables before migration

Blue-Green Database

1. Replicate primary → secondary (green) 2. Apply migration to green 3. Run validation suite against green 4. Switch traffic to green 5. Keep blue as rollback target (N hours) 6. Decommission blue after confidence window

Backfill Strategies

StrategyBest ForInline backfillSmall tables (< 100K rows)Batched backfillMedium tables (100K–10M rows)Background jobLarge tables (10M+ rows)Lazy backfillWhen immediate consistency not required

Batch Processing

DO $$ DECLARE batch_size INT := 1000; rows_updated INT; BEGIN LOOP UPDATE my_table SET new_col = compute_value(old_col) WHERE id IN ( SELECT id FROM my_table WHERE new_col IS NULL LIMIT batch_size FOR UPDATE SKIP LOCKED ); GET DIAGNOSTICS rows_updated = ROW_COUNT; EXIT WHEN rows_updated = 0; PERFORM pg_sleep(0.1); -- throttle to reduce lock pressure COMMIT; END LOOP; END $$;

Dual-Write Period

For expand-contract and parallel change: Dual-write — application writes to both old and new columns/tables Backfill — fill new structure with historical data Verify — assert consistency (row counts, checksums) Cut over — switch reads to new, stop writing to old Cleanup — drop old structure after cool-down period

Test Against Production-Like Data

Never test against empty or synthetic data only Use anonymized production snapshots Match data volume — a migration working on 1K rows may lock on 10M Reproduce edge cases: NULLs, empty strings, max-length, unicode

Migration CI Pipeline

• name: Test migrations
• steps:
• - run: docker compose up -d db
• - run: npm run migrate:up # apply all
• - run: npm run migrate:down # rollback all
• - run: npm run migrate:up # re-apply (idempotency)
• - run: npm run test:integration # validate app
• - run: npm run migrate:status # no pending
• Every migration PR must pass: up → down → up → tests.

Pre-Migration

Tested against production-like data volume Rollback written and tested Backup of affected tables created App code compatible with both old and new schema Execution time benchmarked on staging Lock impact analyzed Replication lag monitoring in place

During Migration

Monitor lock waits and active queries Monitor replication lag Watch for error rate spikes Keep rollback command ready

Post-Migration

Schema matches expected state Integration tests pass against migrated DB Data integrity validated (row counts, checksums) ORM schema / type definitions updated Deprecated structures cleaned up after cool-down Migration documented in team runbook

NEVER Do

NEVER run untested migrations directly in production NEVER drop a column without first removing all application references and deploying NEVER add NOT NULL to a large table without a default value in a single statement NEVER mix schema DDL and data mutations in the same migration file NEVER skip the dual-write phase when renaming columns in a live system NEVER assume migrations are instantaneous — always benchmark on production-scale data NEVER disable foreign key checks to "speed up" migrations in production NEVER deploy application code that depends on a schema change before the migration has completed