The biggest migration I’ve run was a monolithic Drupal 7 application, years past its build date, millions of nodes deep, moving to Drupal 10 — while the site kept publishing the whole time. The client stays unnamed by agreement (the work was agency-side; I was sole technical lead), and the case-study version lives on the work page. This is the part I’d tell another developer over a fire.
The rule that ran the project
Counts are the contract. Every entity type, reconciled source to destination, every single run — not at the end, every run. “The migration looks done” and “the numbers match” are different claims, and the gap between them is where records disappear. When someone asks how you migrate millions of nodes without losing one, the honest answer is boring: you count them constantly, and the count is the definition of done. Everything else follows from making that non-negotiable on day one.
What the defaults would have flattened
A dataset that size isn’t big so much as webbed — deep entity references carrying the relationships the business logic actually runs on. The stock migration paths would have moved the nodes and flattened the web, which produces the most dangerous kind of migration failure: everything present, nothing connected. So the paths were custom, built around the relationships first and the content second.
Increments against a live site
The site couldn’t stop publishing, so the migration ran in passes — the portage method, before I had a name for it. Full carry, then delta runs sweeping up whatever editors created since, repeated until the delta was small enough that the final cutover fit inside minutes. Not a weekend. Minutes. Migrating against a frozen copy is easier and mostly a fantasy; real organizations don’t stop working while you carry their gear.
What survived the project
An untested migration is a rumor, so Behat tests grew around the whole thing — and the suite outlived the migration, becoming the agency’s standard afterward. That’s my favorite kind of outcome: the project paid for its own guardrails, and every migration after inherited them. Millions of nodes, zero lost, and the proof was never a promise — it was a number that matched.