Skip to content
CH SCShop classdrupal · advanced · ~240 min · 10 steps

How to write custom Drupal migration plugins for a D7 migration

The masterclass under the migrate hood: custom source plugins, process pipelines, entity-reference webs, stubs, rollbacks, and the count reconciliation that proves nothing was lost.

August 10, 2026 · by Dane Petersen

The D7 migration lesson covers the trip when core’s migrate paths fit your content. This lesson is for when they don’t — which, on any site with real history, is the moment the actual work starts. Custom content types with field collections, entity references three layers deep, data that lived half in fields and half in some 2013 custom table: the stock paths flatten what your business logic depends on, and the fix is writing your own plugins.

This is the deepest water in Shop Class. You should be comfortable with PHP and have a DDEV copy of both the D7 site and your new Drupal target before starting. Everything here was learned moving a monolithic D7 application with millions of nodes to modern Drupal — without losing a record — and the method scales down just fine.

Understand the three plugin types before writing any

The Migrate API is a pipeline with three stations, and knowing which one your problem belongs to saves days. Source plugins answer “where do rows come from” — a D7 database query, a CSV, an API. Process plugins answer “how does each field value transform in flight” — mapping term IDs, reformatting dates, looking up migrated entities. Destination plugins answer “what gets created” — and you will almost never write one; entity:node and friends cover it. Ninety percent of custom migration code is source plugins and process pipelines. When something feels impossible, you’re usually solving it at the wrong station.

Scaffold the migration module

A migration lives in a plain module. Make one:

mkdir -p web/modules/custom/mymigrate/config/install
mkdir -p web/modules/custom/mymigrate/src/Plugin/migrate/source

mymigrate.info.yml:

name: My Migration
type: module
core_version_requirement: ^10 || ^11
dependencies:
  - drupal:migrate
  - migrate_plus:migrate_plus
  - migrate_tools:migrate_tools

migrate_plus gives you configuration-entity migrations you can edit without cache rebuilds during development; migrate_tools gives you the drush commands you’ll live in: migrate:status, migrate:import, migrate:rollback, migrate:messages.

Define the source database connection

Your D7 database rides alongside the new one. In settings.php (or settings.ddev.php):

$databases['migrate']['default'] = [
  'database' => 'd7',
  'username' => 'db',
  'password' => 'db',
  'host' => 'db',
  'driver' => 'mysql',
  'prefix' => '',
];

In DDEV, import the D7 dump into a second database (ddev import-db --database=d7 --file=../d7-backup.sql.gz). Migrations that declare key: migrate read from it. Never point a migration at the production D7 database — the source should be a copy exactly as old as your last content sync, for reasons that become clear at reconciliation.

Write a source plugin that queries what core can’t

Say D7 stored event registrations in a custom table event_signup with a nid linking to the event node — no entity, no field API, invisible to stock migrations. The source plugin:

namespace Drupal\mymigrate\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;
use Drupal\migrate\Row;

/**
 * @MigrateSource(
 *   id = "event_signup",
 *   source_module = "mymodule"
 * )
 */
class EventSignup extends SqlBase {

  public function query() {
    return $this->select('event_signup', 'es')
      ->fields('es', ['sid', 'nid', 'mail', 'created', 'seats']);
  }

  public function fields() {
    return [
      'sid' => $this->t('Signup ID'),
      'nid' => $this->t('Event node ID'),
      'mail' => $this->t('Registrant email'),
      'created' => $this->t('Signup timestamp'),
      'seats' => $this->t('Seats reserved'),
    ];
  }

  public function getIds() {
    return ['sid' => ['type' => 'integer']];
  }

  public function prepareRow(Row $row) {
    // Derived values belong here, not in twelve process plugins.
    $row->setSourceProperty('mail_domain', substr(strrchr($row->getSourceProperty('mail'), '@'), 1));
    return parent::prepareRow($row);
  }
}

Three things matter more than the rest: getIds() must uniquely identify a row (it feeds the migrate map that makes rollbacks and re-runs possible), query() should push filtering into SQL rather than skipping rows in PHP (skipped rows still count against your totals and poison reconciliation), and prepareRow() is where derived values belong — it keeps the YAML pipeline readable.

Build the process pipeline in YAML

The migration definition, config/install/migrate_plus.migration.event_signups.yml:

id: event_signups
label: Event signups
migration_group: mymigrate
source:
  plugin: event_signup
  key: migrate
process:
  type:
    plugin: default_value
    default_value: signup
  field_event:
    plugin: migration_lookup
    migration: events
    source: nid
  field_email: mail
  field_seats: seats
  created: created
destination:
  plugin: 'entity:node'
migration_dependencies:
  required:
    - events

The load-bearing line is migration_lookup: it translates the old event nid into the new node ID by consulting the events migration’s map table. This is how reference webs survive — every entity reference in your D7 data becomes a migration_lookup against the migration that moved its target. Get the migration_dependencies right and drush runs them in dependency order automatically.

Handle the reference web with stubs and lookups

Real content graphs have cycles — node A references node B which references node A. migration_lookup handles this with stubs: when a lookup misses, it creates a placeholder entity and fills it when the real row arrives. Stubs are correct and safe, with one discipline: never declare a migration finished while its stub count is nonzero. Check for leftovers:

drush migrate:messages event_signups
drush sql:query "SELECT COUNT(*) FROM migrate_map_events WHERE source_row_status = 2"

Status 2 rows are ignored/failed sources; chase every one to a reason you can write down. “Eleven rows failed and I know why — they’re the test events from 2014 with no title” is a finished migration. “Eleven rows failed” is not.

Chain process plugins for the gnarly fields

Process pipelines chain top to bottom, each plugin feeding the next. D7 body text with hardcoded /sites/default/files/ paths, needing a format change and a trim:

  body/value:
    - plugin: str_replace
      source: body_value
      search: '/sites/default/files/'
      replace: '/files/legacy/'
    - plugin: callback
      callable: trim
  body/format:
    plugin: static_map
    source: body_format
    map:
      filtered_html: basic_html
      full_html: full_html
    default_value: basic_html

When a transformation outgrows YAML — conditional logic, multi-field math — write a custom process plugin (src/Plugin/migrate/process/, extend ProcessPluginBase, implement transform()). The rule of thumb: YAML for wiring, PHP for logic. A ten-plugin YAML chain doing surgery is a custom process plugin refusing to be born.

Run in increments, roll back without fear

The development loop that keeps you sane:

drush migrate:import event_signups --limit=50
drush migrate:messages event_signups
drush migrate:rollback event_signups
drush cache:rebuild && drush migrate:import event_signups --limit=50

Rollback works because of the map tables your getIds() feeds — every created entity is tracked against its source row. This is also what makes the live-site pattern possible: run the full import while the old site keeps publishing, then re-run with --update near cutover to sweep up changed rows. The portage method, in drush form.

Reconcile the counts — every entity type, every run

The contract that makes a migration provable instead of promised:

# Source count
drush sql:query --database=migrate "SELECT COUNT(*) FROM event_signup"
# Destination count
drush sql:query "SELECT COUNT(*) FROM node_field_data WHERE type = 'signup'"
# Map accounting: imported + failed + ignored must equal source
drush migrate:status event_signups

Build a tiny script that runs all three for every migration in the group and diffs them, and run it after every import — not at the end. “Looks done” and “numbers match” are different claims, and only one of them survives the client asking where their 2016 registrations went. When source, map, and destination agree — and every discrepancy has a written reason — you’re done, and you can prove it.

Verify like a skeptic before cutover

Counts prove presence; spot-checks prove fidelity. Pull ten random source rows, find their migrated twins, and compare field by field — references resolved, dates in the right timezone, formatted text rendering. Then have someone who knows the content do the same without your map tables open. The migration is ready when the skeptics get bored. If this is the point where you’ve realized your migration is bigger than your appetite — that’s an honest discovery, and it’s exactly what the upgrade assessment prices.

That's the lesson. Back to the shop for more — or if this is the chore your organization never gets to,that's literally what we're for.