I have an interesting situation that I would like feedback from the community on.
I am currently contracting for a company that runs SuiteCRM 8, and has acquired a company that runs SuiteCRM 7. The acquiring company would like to merge all of the important data from the SuiteCRM 7 instance into their SuiteCRM 8 instance.
My question to the community is, has anyone done this before, and what approach did you take that you saw success with?
My initial research hasnāt shown any ready-made tools to perform a merge like this, so I will assume that I will need to develop scripting to merge this data.
Here is a high level overview on how I planned to attack this problem, though I would love feedback from everyone on possible improvements!
Areas of Migration
The following modules are what the organization wishes to merge:
Acquiring organization does not use DocuSign, and intends on letting the subscription lapse, but would like to be able to access signed document information
Perhaps just exporting the signed documents from DocuSign and storing them as documents attached to the contract would work for this?
Field Mappings
While there are similarities between fields in both SuiteCRM instances, there are far more differences than similarities. This is a 10,000 foot view of what I intend the process to be for field mappings:
Document 1 for 1 mappings
Document mappings with the same field name, but that have different drop-down values
Document what drop-down items need to be added in the SuiteCRM 8 instance
Add those items
Document what drop-down items in the SuiteCRM 7 instance need to be converted to similar drop-down items in the SuiteCRM 8 instance
Create a dictionary of these changes in the migration script
Document fields that do not exist in the SuiteCRM 8 instance that will need to be created
Create those fields
Document fields that need to be transferred, but that will have a new field name on the SuiteCRM 8 instance
If required, create those fields
Document fields that do not need to be transferred
Migration Method
Initially, a test CRM environment will be created from a clone of the SuiteCRM 8 server to test the migration with. Once acceptable results are seen, the data will be migrated to the live instance of SuiteCRM 8
As both SuiteCRM instances support API V8 I intend to write a script using Python and PySuiteCRM to perform the migration. The high level logic of the script will be as follows:
Query all records and relationships for required modules from SuiteCRM 7 instance into a local sqlite database (so the record insertion can be restarted if a crash occurs)
Store field data as JSON
Store relationship data as JSON
Perform API āUPSERTā of all records (excluding relationships)
An āUPSERTā will ensure that if a record already exists, a duplicate record insertion will not be attempted.
Once all records have been created, perform API āUPSERTā of all record relationships
For Contracts, convert all DocuSign records into PDF Documents, related to the contracts they are associated with
Once all records have been merged, use SFTP to transfer all uploaded files from SuiteCRM 7 to SuiteCRM 8.
Suggestions?
I am sure there are some things that I havenāt thought about or have just straight up missed, so I would greatly appreciate feedback or suggestions on what I should do to improve my methodology.
sounds like a fairly standard CRM migration / merge project (whether the source / 2nd CRM is also Suite isnāt that important - the concept applies for Salesforce, Hubspot, Zoho etc. as well).
Field mapping will be your best friend here.
Just like you outlined it - a list of all the target fields + the gaps / non existing / to transform / not to migrate fields.
What Iād change is the Python approach.
Usually my go to language as well for many things, but give n8n a try.
The nice thing is, that youāre building out your field mapping visually with drag n drop.
Additionally the āloggingā is visually and instant - so you donāt need to produce logs from your code and switch back and forth.
Iād only use the SQLite if you have a fairly complex data transformation / pipeline going on with silver records and a golden record + final load - possible from multiple source systems.
The API is usually so solid, thatād Iād to a direct 1:1 migration with some n8n workflows.
The Upsert mechanisms make the whole logic self healing - so you can just restart the whole migration logic.
Check out my n8n videos for SuiteCRM if youāre interested:
how they will be managed in the acquiring company? Please check the hierarchy based on Roles and Security Group
The second one is Accounts:
how are they manage in the acquiring Company?
Which is the Customer Journey? Are they using Accounts only for effective Customers (ERP like approach) or for both Potentials and Effectives?
PS: i agree with BastianHammer. Itās the same flow youād follow for other CRM migrations. An important step of your job will be surely the Field Mapping
The acquiring company made it relatively easy, in that they will not be bringing any processes over from the acquired company, accounts and employees are completely migrated to their internal process, so I donāt have to implement any additional custom logic or security roles/groups to account for that as it is already in their existing SuiteCRM instance (though there will be additional feature development after the merge is complete).
They require that all data be brought over, whether itās current customers, lost customers, active leads, dead leads, they want everything.
Honestly, when I do a migration, I find it easier just to export each module to CSV, organize/edit it the way I want then import each module via the standard import function. Itās clean and easy. You just have to do it in a specific order like users first, then accounts, then contacts, then leads, then activities and so on. By the time you develop a python script and test it, you could be done already.
I need to make sure I maintain all relationships as well as documents/files that are associated with those records. This is a pretty large instance with multiple thousands of each type of record.
I am focusing on learning n8n to perform the migration, as there are going to be several automations the client desires after the merge is complete that would be best suited for something like n8n anyhowā¦
The acquiring company arenāt using some of the modules that are coming from the acquired company (like cases), but they are ok with migrating the data one-to-one so they can maintain the records, so yes I am going to focus on field mapping.
@joshpatten its not a problem how many thousands of records. Each record has a relationship tied to it Just import it will keep all the relationships (one to many).
The only caveat is many to many relationships, they generally have a third table to relate them. Those are a little more complicated but pretty easy to. Just import the entire relationship table via phpmyadmin. Since via import you keep all the IDās it all works. Done it many times.
You can map field ā field right in the import wizard. Yes you have to kind of know in advance what to equate it to but importing the entire moduleās records via import makes this super easy.