I have a project that involves a nightly importing of data from an external source, and from what I've seen Migrate module would be a more robust solution that me writing code. I've been looking at it but haven't been able to figure out how to implement it to fit my needs, so I'm hoping someone here might be able to shed some light on it for me.
The tool I'm building is a middleware tool for a bike shop to import inventory data from a provider, add some data to it, and then export it out to a CSV file. I already have a drush command to import the data from the external source (a SQL Server db) into MySQL. The issue I'm trying to get worked out is getting the data from my custom table into Drupal nodes. I wrote some custom code to do it, but it seems like using Migrate would be a lot cleaner and more robust.
Every night when I update my local copy of the data, I need to determine three sets of data in comparison to the previous import:
1) what's new
2) what is no longer there
3) what has changed
This also includes getting an image from the external server based on a path in the imported data. Once I have the records identified, I need to import them into Drupal nodes. There are two ways I track the status of each node:
1) a CCK field called Status that has three possible values: new, change, delete
2) A log table that records each status change.
So if a record is:
new - I need to create a new node with a status of New
deleted - edit existing node, change Status to Delete
changed - edit existing node, change Status to Change
and in all three cases, update the log table (I then display that log data in each node).
I can write hooks to do the Status field and log updates. The biggest issue I have is getting the data into nodes, either creating or updating, and doing it in an automated fashion. From what I've seen, Migrate should be able to handle it fine, but it looks like a pretty steep learning curve. If I'm seeing correctly, it's all through code and not a UI, and there doesn't seem to be much (at least up to date) good documentation (migrate.api.php is empty). Any light that could be shed on this for me would be greatly appreciated.
Thanks.
