Issues importing opportunities

Hi !
I am importing all my date from SugarCRM to SuiteCRM. I successfully imported all my accounts and contacts, but I’m having issues importing Opportunities.
I exported all the date from Sugar into a CSV, and I use the import functionality of Suite to import the data.
When I import my csv, the importation stops after a few records, and this error is displayed. Also, it does not redirect me, so I can’t cancel the importation. So it imports only a few of the records, and it creates duplicates, even though my data hava id, so it should not be able to import twice the same record. I’m guessing it ignores the id.
I Don’t know which record drives the error, since it does import a few records before failing.

I tryed importing only 1 record, and it successfully imports it, but still displays the error,
and does not redirect me to the next page.

image

The line it refers to is linked to a bug fix :

    /*
    * Bug 34854: Added all conditions besides the empty check on date modified.
    */
    if ((!empty($focus->new_with_id) && !empty($focus->date_modified)) ||
         (empty($focus->new_with_id) && $timedate->to_db($focus->date_modified) != $timedate->to_db($timedate->to_display_date_time($focus->fetched_row['date_modified'])))
    ) {
        $focus->update_date_modified = false;
    }

I tryed removing it, but if I do so it ignores the ids and creates duplicates of all the data already imported…

Should I delete all that’s already been imported and reimport everything after removing the line ?

Thanks for your help :smile:

Hey,

There might be a few things worth checking

What is your setup like?
(ie, Which PHP version? MySQL/MSSQL version?)

I found a previous thread in which similar error messages appear for a user using PHP 7.4:

Which I don’t believe is fully supported, as per the compatibility matrix:

In the same thread above, a user mentions modifying the error_reporting value in the php.ini
Does this resolve the issue?

If not, you could indeed delete the data, and re-import with the change you suggested
It’s not ideal that it ignores IDs, but if it works for now as a workaround, then at least you can get the data in the CRM!

If you’re seeing PHP errors on your screen, you need to turn off display_errors in your php.ini.

This is not just an aesthetic issue - it breaks screens, there’s no point in trying to troublshoot any other functionality fault until after that is fixed.

A slightly different thought approach to dealing with this could be:

  • to create a copy of the import file, and then view the file to be imported in a spreadsheet. If you delete all of the dates modified, then the code that is run during the import will never touch the code that is giving you that NOTICE.
  • You have an id associated with each of the new imports - the same code block is only run if there is an id associated with the data being imported. (potentially you might remove the ID in the imported data?
  • In order to avoid duplicates being imported, then the ID IS REQUIRED, but assuming that all of the imported data does have an ID associated with it then you can select only import new opportunities?

Trying to get greater visibility of the opportunities that are causing problems can be gained by adding additional logging to the code. The reference document for this is here:

You might wish to insert something like:

Blockquote
/*
* Bug 34854: Added all conditions besides the empty check on date modified.
*/
if ((!empty($focus->new_with_id) && !empty($focus->date_modified)) ||
(empty($focus->new_with_id) && $timedate->to_db($focus->date_modified) != $timedate->to_db($timedate->to_display_date_time($focus->fetched_row[‘date_modified’])))
) {
LoggerManager::getLogger()->notice('ID is: '.$focus->new_with_id);
$focus->update_date_modified = false;
}

You can then look at the suitecrm.log files in order to identify which imports are causing issues.

Importing opportunities is painfully annoying in my experience with some of these ‘glitches’. Unfortunately many users, and indeed many 3rd party modules add extra fields that aren’t quite correctly configured. My usual approach is that for these columns I either ‘Do not map’ or sometimes have to set all of them to True/Y/1 and I never seem to know which is the correct import term until I try it.

Looks like he @roxa may be using 7.4.7 based on their post here:

The compatibility matrix does seem to imply that 7.4.7 is not yet supported?..but is that because the compatibility matrix:

is out of date?

Ok so I changed the error_reporting value in my php.ini to :

error_reporting = E_ALL & ~E_NOTICE & ~E_DEPRECATED & ~E_STRICT & ~E_WARNING

Looks like it solved the issue, I’ll see if it works for all my data.

Do you think I should still change my PHP version ? Could there be others issues because of that ?

Anyway, thanks for your help :smiley:

I just realised it is still ignoring the ids, and I think I will need them to link the Opportunities to other records…
I’m going to downgrade my PHP then :slightly_frowning_face:

Yes - unfortunately, you need to identify the IDs as that’s what links the bits together :frowning_face:

Downgrading PHP to something inside the compatibility Matrix is mandatory unless you are just testing on a demo server and you like to take risks

Turning off display_errors keeps errors from going to the screen, makes them go only to the logs. I really recommend that you do this.

Changing error_reporting completely suppresses certain kinds of errors from everywhere, including the logs, which I don’t really recommend…

Ok, so I downgraded my PHP version to 7.3, and it works better, ids are imported now.

My display_errors has Always been set to “Off”, the issue was with an error in a popup, that apparently stopped the redirecting to the next page.

Thank you

1 Like

Just one more thing,

After my import is complete, there are links to get files containing the non-imported records, but these send me to a “HTTP 501 error”…

My log shows the lines :

Fri Aug 7 15:04:25 2020 [5732][1][WARN] CSS File Dawn/yui.css not found
Fri Aug 7 15:04:26 2020 [5732][1][ERROR] File upload://IMPORT_Opportunity_1-0 cannot be touched
Fri Aug 7 15:04:26 2020 [5732][1][ERROR] File upload://import/dupes_1.csv cannot be touched
Fri Aug 7 15:04:26 2020 [5732][1][ERROR] File upload://import/dupesdisplay_1.csv cannot be touched
Fri Aug 7 15:04:26 2020 [5732][1][ERROR] File upload://import/errorrecords_1.csv cannot be touched
Fri Aug 7 15:04:26 2020 [5732][1][ERROR] File upload://import/errorrecordsonly_1.csv cannot be touched
Fri Aug 7 15:04:26 2020 [5732][1][ERROR] File upload://IMPORT_Opportunity_1-1 cannot be touched
Fri Aug 7 15:04:36 2020 [6216][1][FATAL] Caught error: stat(): stat failed for upload/import/error_1.csv
Fri Aug 7 15:04:36 2020 [6216][1][FATAL] Caught error: touch(): UploadStream::stream_metadata is not implemented!
Fri Aug 7 15:04:36 2020 [6216][1][ERROR] File upload://import/error_1.csv cannot be touched
Fri Aug 7 15:04:41 2020 [6216][1][FATAL] Caught error: stat(): stat failed for upload/import/status_1.csv
Fri Aug 7 15:04:41 2020 [6216][1][FATAL] Caught error: touch(): UploadStream::stream_metadata is not implemented!
Fri Aug 7 15:04:41 2020 [6216][1][ERROR] File upload://import/status_1.csv cannot be touched

Any Idea what this coukld be caused by or how to fix it ? Not that it is vital, but it could be usefull not to cancel the whole import because of a few errors :smile:

Hi ! I’m having new issues when trying to update existing records.
It works for a few records, and then prompts this error, and stops the import.
This happens for every module I try importing data into.

Hi,
pgr was already mentioning this:

=> display_errors=Off

E: you probably need to restart the service afterwards

My display_errors is set to “Off” (and has Always been set to off).
The error I’m referring to does not appear in the window but in a popup…

Since you’re still getting warnings: did you modify the correct php.ini file? You could verify this using a simple phpinfo() debug page.

I’m modifying the file indicated in my PHP manager, there :

The phpinfo() seems to have the right value…