I just finished battling the windmills of Drupal Migrate (6 to 7) File Migrations, and perhaps unlike Don Quixote, I managed to win.
I first have to say that I love and have used Acquia's site-upgrade tool for the d6 to d7 migration, and it was awesome.
In my recent experience for the drupal 6 to drupal 7 realm, this is no longer the case. Too much time has passed. I had many iterations of failure upon failure. Hacking and patching along the way. I had to cut bait.
I am saying this as a developer who has experience upgrading over 5 sites successfully using Acquia's site-upgrade for the d6-d7 migration, and as someone who loved using it. I'm not saying it's dead, but it was dead for this project in terms of time expenditure and budget.
I’m not going to be long here.
In addition to presenting the essential pattern that was the magic combination of things for me, I want to present the information from the standpoint of avoiding pain.
1. Less is more.
I used the Migrate module with the Migrate Drupal 2 Drupal (migrate_d2d) module to extend classes in my own migrate module. D2D has graciously taken the initiative to solve the general handshake between different versions of drupal. You only want to give it what it wants and the very minimum with respect to File migrations and with respect to extending classes.
You do not have to redesign the basic handshake, you just need to give it the extras that are custom to your process. If you start overriding too many variables, especially without understanding them, you can run into issues.
2. Learn to test in little chunks
There are two ways this comes into play. The first is that there is a --limit=10 that will tell the migration module not to bring in the entire set, but only, for example, 10. This enables you to test quickly without having to wait for a whole set of a class to migrate.
3. Learn and use the drush commands.
For example, when I accidentally forgot to set a limit and needed to start over I chose would first interrupt the process and then, with a basic pattern of:
drush mxxx –limit=10 YourClass.
The sequence of ‘mxxx’ that worked best for me is: 0) mst (stop), 1) mrs (reset), 2) mr (rollback), and then you can 3) mi (import) when you are ready to import.
drush mreg –limit=10 YourClass is for when you have done changes that you want migrate to reconsider.
drush ma –limit=10 YourClass shows you an analysis of your Classes which is super helpful.
4. Build scaffolding and start over often.
What this means from a functional perspective is that when you get to a good place to rest, save a version of the database so that you can go back to it if you run into problems or glitches testing.
The most important of these backups is to a state right AFTER you install migrate and migrate_d2d, and BEFORE your custom migrate module is enabled.
Why?
Because if you are making specific changes, particularly to the ‘.migrate.inc’ file that defines some of your $arguments and the migration API. If you make changes there, TRUST ME, it is less of a waste of time to delete/create a new database and then overwrite that database with your saved one and then enable your custom module from scratch then it is to try and realize that you have exceeded the modules ability to give you accurate information about whether the whole package of your module is working together.
Otherwise, you will spend hours testing, get silent fails inaccurately, reload the module and find that all of your definitions are different than you thought they were last time you ran the analysis, and then wonder if it’s less painful to die jumping off a bridge or with an overdose of pills.
That said, here is the basic pattern for files that worked for me:
5. Basic Pattern
Well first off, I lit plenty of candles throughout the process. Here are some fundamental and important code patterns:
0) In your .migrate.inc file where you define your $arguments:
/**
* Each migration being registered takes an array of arguments, some required
* and some optional. Start with the common arguments required by all - the
* source_connection (connection key, set up in settings.php, pointing to
* the Drupal 6 database), source_version (major version of Drupal), and
* group_name (a.k.a. import job).
*/
// TODO: DEPENDS ON ENV
$common_arguments = array(
'source_connection' => 'legacy',
'destination_connection' => 'default',
'source_version' => 6,
'destination_version' => 7,
'source_database' => array(
'driver' => 'mysql',
'database' => 'jello',
'username' => 'jello', // Ideally this user has readonly access
// Best practice: /*use a variable (defined by setting $conf in settings.php, or
// with drush vset) for the password rather than exposing it in the code.
//'password' => variable_get('example_migrate_password', ''),
'password' => 'yourpassword',
'host' => 'localhost',
'prefix' => '',
),
'destination_database' => array(
'driver' => 'mysql',
'database' => 'jello7',
'username' => 'jello', // Ideally this user has readonly access
// Best practice: /*use a variable (defined by setting $conf in settings.php, or
// with drush vset) for the password rather than exposing it in the code.
//'password' => variable_get('example_migrate_password', ''),
'password' => 'yourpassword',
'host' => 'localhost',
'prefix' => '',
),
'format_mappings' => array(
'5' => 'markdown',
),
Remember that you will want to be defining two databases in your settings.php file or your settings.local.php file. In the case of the example above, I define the 'legacy' database in the same array as my 'default' database.
$databases = array (
'default' => array (
'default' => array (
'driver' => 'mysql',
'username' => 'jello',
'password' => 'yourpassword',
'port' => '',
'host' => '127.0.0.1',
'database' => 'jello7',
'charset' => 'utf8',
'collation' => 'utf8_general_ci',
),
),
'legacy' => array(
'default' => array(
'database' => 'jello',
'username' => 'jello',
'password' => 'yourpassword',
'host' => '127.0.0.1',
'port' => '',
'driver' => 'mysql',
'charset' => 'utf8',
'collation' => 'utf8_general_ci',
),
),
);
1) In your node class where you have the image or file field:
// Images.
$this->addFieldMapping('field_image_file', 'field_image_file')
->sourceMigration('JelloFile');
$this->addFieldMapping('field_image_file:file_class')->defaultValue('MigrateFileFid');
$this->addFieldMapping('field_image_file:preserve_files', NULL, FALSE)->defaultValue(TRUE);
//$this->addFieldMapping('field_image_file:language')->defaultValue(LANGUAGE_NONE);
$this->addFieldMapping('field_image_file:alt', NULL)->defaultValue('Image');
$this->addFieldMapping('field_image_file:title', NULL)->defaultValue('Image');
2) Your File class:
/**
* Migrate Files to D7.
*/
class JelloFile extends DrupalFile6Migration {
protected $legacyPath;
public function __construct(array $arguments) {
$arguments['machine_name'] = 'JelloFile';
parent::__construct($arguments);
$this->addFieldMapping('pathauto', NULL, FALSE)->defaultValue(0);
$this->addFieldMapping('urlencode', NULL, FALSE)->defaultValue(TRUE);
$this->addFieldMapping('destination_file', 'filename', FALSE);
if (module_exists('path')) {
$this->addFieldMapping('path', 'filepath', FALSE);
}
}
3) And then the order that they get placed along with the migration commands is in the profile install file like this:
// A COMPLETE MIGRATION SET FOLLOWS. Some classes are not included here and that is on purpose.
// The jello-user group.
drush_migrate_import('JelloRole');
drush_migrate_import('JelloUser');
// The jello-taxonomy-complex group.
drush_migrate_import('Regions');
drush_migrate_import('Microsites');
// The jello-file group.
// Has to be imported from drush or from code or from here. Not visible in UI.
drush_migrate_import('JelloFile');
// The jello-taxonomy-complex group.
drush_migrate_import('Topics');
// One from the jello-content-complex group
drush_migrate_import('Image');
// The jello-taxonomy group.
drush_migrate_import('Featured');
drush_migrate_import('JelloTopics');
drush_migrate_import('Language');
drush_migrate_import('NewsAuthor');
drush_migrate_import('NewsSource');
drush_migrate_import('OccupationalGroups');
drush_migrate_import('PolicyAreas');
drush_migrate_import('PublicationAuthor');
drush_migrate_import('PublicationType');
drush_migrate_import('ReportAuthor');
drush_migrate_import('ResourceAuthor');
drush_migrate_import('ResourceTypes');
drush_migrate_import('Tags');
drush_migrate_import('Theme');
// Couple from the jello-content-complex group
drush_migrate_import('Publications');
drush_migrate_import('Specialist');
// The jello-content group.
drush_migrate_import('Events');
drush_migrate_import('Page');
drush_migrate_import('PublicationsResources');
// The jello-content-complex group.
drush_migrate_import('Blog');
drush_migrate_import('DdblockNewsItem');
drush_migrate_import('Event');
drush_migrate_import('OccupationalGroupPage');
drush_migrate_import('ProgramPage');
drush_migrate_import('Reports');
drush_migrate_import('Resources');
If you have a class that uses something in another class, you want to migrate those classes first.
There are many other patterns and I have created a sample migration that uses them attached to this blog.
That repository has been hosted at github until they decided to move to the dark side of Micro$oft. So the meantime I am hosting it here: https://gitlab.com/jellobrain/jello_migrate. To easily learn how to make the move yourself go here.
Enjoy!