Stars: 165
Forks: 8
Pull Requests: 18
Issues: 20
Watchers: 9
Last Updated: 2020-06-01 08:35:02
🔀 UXDM helps developers migrate data from one system or format to another.
License: GNU Lesser General Public License v3.0
Languages: PHP
UXDM helps developers migrate data from one system or format to another.
UXDM can be easily installed using Composer. Just run the following command from the root of your project.
composer require divineomega/uxdm
If you have never used the Composer dependency manager before, head to the Composer website for more information on how to get started.
Create a new PHP file to contain your UXDM migration code. In this example, we'll call it user-csv-import.php
. Remember to add require 'vendor/autoload.php'
and relevant use
statements, if necessary.
Create your source and destination objects. This example uses a CSV source and PDO (database) destination.
$csvSource = new CSVSource('users.csv');
$pdoDestination = new PDODestination(new PDO('mysql:dbname=test-database;host=127.0.0.1', 'root', 'password'), 'users');
$migrator = new Migrator;
$migrator->setSource($csvSource)
->setDestination($pdoDestination)
->setFieldsToMigrate(['id', 'email', 'name'])
->setKeyFields(['id'])
->withProgressBar()
->migrate();
php user-csv-import.php
from the command line and will get a nice progress bar.See the sections below for more information on the available source and destination objects, and more advanced usage examples.
Each UXDM migration requires a source object and at least one destination object. These determine where and how data is read and written. The UXDM package works with a variety of source and destination objects, including the following.
Some of these are built-in to the core UXDM package, while others are available as separate packages.
Source and destination objects can be used in any combination. Data can be migrated from a CSV and inserted into a database, just as easily as data can be migrated from a database to a CSV.
You can also use similar source and destination objects in the same migration. For example, a common use of UXDM is to use a PDO source and PDO destination to transfer data from one database to another.
Please see the Sources & Destinations page for more sources and destinations, and detailed documentation on their usage.
An example of a basic database to database UXDM migration is shown below.
$pdoSource = new PDOSource(new PDO('mysql:dbname=old-test;host=127.0.0.1', 'root', 'password123'), 'users');
$pdoDestination = new PDODestination(new PDO('mysql:dbname=new-test;host=127.0.0.1', 'root', 'password456'), 'new_users');
$migrator = new Migrator;
$migrator->setSource($pdoSource)
->setDestination($pdoDestination)
->setFieldsToMigrate(['id', 'email', 'name'])
->setKeyFields(['id'])
->withProgressBar()
->migrate();
This migration will move the id
, email
and name
fields from the the users
table in the old-test
database, to the new_users
table in the new-test
database, replacing any existing records with the same id
(the key field).
You can use UXDM to validate the source data. If validation fails part way through a migration, the migration will
halt and a ValidationException
will be thrown. However, if ->validateBeforeMigrating()
is called, all data rows
will be preemptively validated before the migration begins.
The code below shows how to validate various fields.
$pdoSource = new PDOSource(new PDO('mysql:dbname=old-test;host=127.0.0.1', 'root', 'password123'), 'users');
$pdoDestination = new PDODestination(new PDO('mysql:dbname=new-test;host=127.0.0.1', 'root', 'password456'), 'new_users');
$migrator = new Migrator;
$migrator->setSource($pdoSource)
->setDestination($pdoDestination)
->setFieldsToMigrate(['id', 'email', 'name'])
->setValidationRules([
'id' => [new Required(), new IsNumeric()],
'email' => [new Required(), new IsString(), new IsEmail()],
'name' => [new Required(), new IsString()],
])
// ->validateBeforeMigrating()
->setKeyFields(['id'])
->withProgressBar()
->migrate();
This migration will validate the source data matches the defined validation rules.
UXDM uses the Omega Validator package. See its documentation for all available validation rules.
This examples shows how UXDM can map field names from source to destination.
$migrator = new Migrator;
$migrator->setSource($pdoSource)
->setDestination($pdoDestination)
->setFieldsToMigrate(['id', 'email', 'name'])
->setKeyFields(['id'])
->setFieldMap(['name' => 'full_name'])
->withProgressBar()
->migrate();
This migration will move data from the source name
field into the destination full_name
field, while still moving the id
and email
fields normally.
Sometimes the data you want to move from source to destination needs transforming. This can be changing existing items of data, adding new data items, or removing items you do not need.
UXDM allows you to create one or more transformer objects, and add them to the migration. See the following examples of how to use transformers to manipulate your data.
This example shows how you can transform existing data items during migration.
class NameCaseTransformer implements TransformerInterface
{
public function transform(DataRow $dataRow): void
{
$nameDataItem = $dataRow->getDataItemByFieldName('name');
$nameDataItem->value = ucwords(strtolower($nameDataItem->value));
}
}
$migrator = new Migrator;
$migrator->setSource($pdoSource)
->setDestination($pdoDestination)
->setFieldsToMigrate(['id', 'email', 'name'])
->setKeyFields(['id'])
->addTransformer(new NameCaseTransformer())
->withProgressBar()
->migrate();
This migration will ensure that all names fields have consistent case.
This example shows how you can add new data items while the migration is taking place.
class AddRandomNumberTransformer implements TransformerInterface
{
public function transform(DataRow &$dataRow): void
{
$dataRow->addDataItem(new DataItem('random_number', rand(1,1000)));
}
}
$migrator = new Migrator;
$migrator->setSource($pdoSource)
->setDestination($pdoDestination)
->setFieldsToMigrate(['id', 'email', 'name'])
->setKeyFields(['id'])
->addTransformer(new AddRandomNumberTransformer())
->withProgressBar()
->migrate();
This migration will add a random number into a field called random_number
for each row of data.
This will then be migrated to the destination database along with the other fields.
This example demonstrates how data items can be removed from a data row. You may wish to do this if you want to use its value, but not actually migrate it to the destination.
class EmailToHashTransformer implements TransformerInterface
{
public function transform(DataRow $dataRow): void
{
$emailDataItem = $dataRow->getDataItemByFieldName('email');
$dataRow->addDataItem(new DataItem('email_hash', md5($emailDataItem->value)));
$dataRow->removeDataItem($emailDataItem);
}
}
$migrator = new Migrator;
$migrator->setSource($pdoSource)
->setDestination($pdoDestination)
->setFieldsToMigrate(['id', 'email', 'name'])
->setKeyFields(['id'])
->addTransformer(new EmailToHashTransformer())
->withProgressBar()
->migrate();
This migration gets the data from the email
field in the source, creates a
new email_hash
data item which contains an md5 of the email address, and then
removes the original email
data item. This new email_hash
will then be
migrated to the destination database along with the other fields, excluding
the removed email
field.