We have a SuiteCRM application which we have customised by using Studio to add new fields to Accounts and Contacts - all fine.
We now wish to develop a specialised version of this application to meet the requirements of a particular user. Therefore we need to copy our application, including its customisations, but start with a fresh database. To this end, we have followed the procedure suggested in postings #39348 and following (“How to change database settings for a cloned SuiteCRM”), i.e. carried out a new “out of the box” 7.4.3 installation, then immediately wiped that application, copied over it our existing application, and tweaked the necessary config setings. In other words, we were using the installation purely for the purpose of creating a new database containing whatever data a new application needs to have (I am assuming a new SuiteCRM database is not completely empty, but contains some basic data to get things working).
The Accounts and Contacts detail views will not load because they contain the aforementioned custom fields (they loaded when we reverted to the default view), and on further investigation, those fields are not visible in Studio (on the lines where those fields are shown in the old application, in the new one there is simply the word “null” in the “Name” column). The custom fields are still present in the custom vardefs.ext.php file.
Therefore, I am assuming that the custom fields are not properly visible because (as I can see, using SQL) they do not exist in the database. I have tried the “Quick repair and rebuild”, but this has no effect.
It would appear, therefore, that I need to manually add the necessary columns to accounts_cstm and contacts_cstm using SQL before the custom fields will become visible. Have I done something wrong in the procedure I have followed? Or is there a better way of cloning a SuiteCRM instance? This is surely a common scenario which developers face. Any guidance greatly appreciated!
When you tried the “Quick Repair and Rebuild”, did you scroll down all the way to the bottom? There should be a button there to “sync database with vardefs” or something similar.
Thankyou, but no, I can see no such button on the “Repair” page. When I click on “Quick Repair and Rebuild”, the resulting output says at the bottom “Database tables are synced with vardefs”, but still those fields are missing from the database, whilst they are present in the file custom/modules/Accounts/Ext/Vardefs/vardefs.ext.php, which had been copied from the original application
that is a generated file; but rather the original file from which it is generated.
I think in general, files under directories called
custom/modules/*/Ext/*
are generated.
So if you copy that one it can be overriden by the Quick Repair and Rebuild. Do you have Jim Mackin’s eBook “SuiteCRM for Developers”? You can check there which customization go in which directories.
Also, you might use this advanced technique to track which files SuiteCRM is looking for.
BTW, if that looks like too much trouble, I believe an easy path could be:
export accounts_cstm and contacts_cstm (structure only) from the old database
import them in the new one
I don’t know if this will fix things, because I really think that if the vardefs had your fields where they should, SuiteCRM would offer to create the tables for you automatically. But it won’t hurt to try…
Thankyou again. Yes, I have bought the Jim Mackin book.
As regards copying the “original” files, rather then the “generated” ones, as I said, I copied the whole application. I did not know where all the files were, because I had originally used Studio to add the custom fields. Having run a recursive “grep” of the whole application for one of the field names in question, I see that there are relevant files in custom/Extension/modules/Accounts/Ext/Vardefs.
I can easily enough check and fix the database discrepancies using mysql/mysqldump and hopefully that will fix the problem for the time being.
I manually added the custom fields using SQL, but this made no difference - the custom fields are not recognized when loading the views, or when going into Studio. I even started from scratch, i.e. copied the whole application again. I now have an exact copy of the application (with site_url, unique key etc changed), and an exact copy of the database structure (used mysqldump and diff to verify), but still plagued by the same problem!
But here’s the thing - on a whim, I changed the config so that it pointed at the old database. Bingo! Everything works as before! No doubt I could have dumped the old database and reloaded it into a new one, and that would have worked as well (that is what I do anyway with no problem when I want to create a test system with relevant data in it).
S0 - if the application FILES are the same, and the database STRUCTURE is the same, but the system runs properly with a current database but NOT with a fresh database, even though the fresh database has had the missing custom fields added to it to mirror the structure of the current database, does not this suggest that the problem is not with the database STRUCTURE, but the database CONTENT, i.e. there exists something along the lines of a “my custom fields” table, the CONTENTS of which I need to copy from the old database to the new one?
Are you sure your customizations are only custom fields? If you created any relationships, you need to copy those tables over too (and this fits the description of your diagnosis).
If you search in phpmyadmin for tables including the number “1”, you can see the custom relationships created (if the default names were used, they end in “1_c”… if not, it’s harder to find them…)
In my system this is what I see:
So these are the custom relationships that ship with SuiteCRM (left-overs from when some SalesAgility modules were not integrated into the core, I believe). If you have any others, consider moving them to the new server.
… or even just “show tables like ‘%1%’;” ! No, we have not created any relationships so far, just added custom fields. Could the table I need be “fields_meta_data”?
Yes - back in business. Thanks for your help. “fields_meta_data” it was; SQL-ed from the old database those records that were missing from the new one, and the fields now magically re-appear!
My help was just chatting with you until you found the solution…
After the migration is complete you should seriously consider an upgrade. Newer PHP (7.0 works fine with SuiteCRM, much faster than previous versions) and newer SuiteCRM (maybe 7.8.6 is the right one).
Yes - I am keen to upgrade as soon as possible; at the moment I am constrained by the version of the operating system (and hence PHP version) on our server, which has other compatibility constraints, but now we are devoting more attention to web applications, I hope soon to have the luxury of an independent server for web development, which will permit such an upgrade.
I know it’s an old topic but having the same problem. So I’m assuming OP’s initial DB dump wasn’t really done properly? Otherwise why would he have to:
"fields_meta_data” it was; SQL-ed from the old database those records that were missing from the new one, and the fields now magically re-appear!"
I’m guessing in my case the dump also didn’t work properly because i’m missing a bunch of fields that should be in fields_meta_data apparently… I tried deleting the generated vardefs to see if it will force the system to rebuild the fields, but it didn’t help. Will see if anything else can be done, since I only have the DB dump to work with
Nope, same suitecrm version also. I’m sure I copied the same exact files and did a complete DB dump. So everything should be as it was on the previous machine.
When you copy the whole database, and all the files, and migrate to the same version, it should just work
All I can think of is running a QR&R watching the log for messages, and also a Repair relationships. These processes make the database tables (fields_meta_data and relationships) match the files, I think.
You’re right. Not sure why the mysql dump did not move this table also, I see a bunch of other people have the same problem (may be a problem with mysql dump?).
Thank you for the suggestions: I tried doing repair relationships and did QR&R in the logs I see the following message for one of the missing fields:
Wed Dec 18 19:14:10 2019 [3604][1][DEBUG] ModuleInstaller->merge_files(): found custom/Extension/modules/external_brokers_commissions/Ext/Vardefs/sugarfield_single_amount_c.php
But not sure what it means, it’s the only mention of the missing field I see.
I don’t believe this can be a bug with MySQL Dump, and I don’t think this is common. It just happens occasionally and I suspect it’s only a problem when something got messed up before the migration… some inconsistency somewhere…
