SuiteCRM8 - Failed to show records after data import

Hi,
After imports of 8000 ‘Accounts’ records, dose not show list view its seem stuck and showing progress icon continuously.
CPU usage high on browser PC, and server site is normal.

-Fareed Ahmed.

Hey @fareedkh !

Awesome you’re exploring SuiteCRM 8.

Are you able to look at the suitecrm.logs and see if there is anything obvious going on.
At what point is the page hanging? After the whole importing process or mid way through?

Hi,

Thank you for responding.
The import has been completed successfully,
After clicking on view, it begins processing with the suitecrm progress icon and does not return,
and there are no errors in any log files.

Hmm,

Importing is unfortunately synchronous so it doesn’t have a background job and will consume the front activity even after it’s stated until it’s completed. We need to confirm that the importing has definitely completed and I believe we can do that by looking at the DEBUG log potentially. Is your logger level set at Fatal (the default), try turning it on to DEBUG and see the suitecrm.log output. There will be alot generally, but hopefully it will indicate that the importing is still executing.

Might be worthwhile to check the php.ini parameters that might be relevant to getting a large job finished

max_execution_time
upload_max_filesize
post_max_size
memory_limit

restart web server after any changes.

Hi ,
now ‘Account’ Module showing records in the ‘Detail View’.
just remove ‘ID’ filed from CSV import file and re-import.

but in my case ‘ID’ is important to relate the records of other modules.

So, looking into this.

-fareedkh.

Maybe the problem is with a specific ID in a specific row? You might have a repeating ID, which the database rejects because it’s a unique key field.

You might have clues (MySQL errors) in the php_errors.log

I hook into this.
We have set up a new v8 site with converted data from v6. Work ok.
But Accounts is a problem as stated in post here. List view is very slow and ‘hangs’. Not many records.
That is in computer access from Linux and Windows clients. But running from Android phone working fine. List start shows up in a few seconds.

can 7.12.2 be imported to 8? If yes, where are the specs. THX

Another test session today. Telling me the same thing. Accounts list data showing up VERY slow accessed from my Linux box (2 minutes) but very good from my Andriod phone (2 seconds).
That is telling me that the database is responding OK, assuming the DB calls are the same in both cases. So where is the sink ?

Hi @goingsmart,

The upgrade path from 7.x to 8.x hasn’t been released yet.

Though the data layer should be almost the same between the 2.

Hi @Kenta,

Hm… that is very strange. On desktop browser, if you look into the network tab in the dev tools, do you have many re-directs or something similar?

The app is the same regardless of the browser, so my guess is that there maybe something network related.

Another this related with performance. Do you have opcache setup? that makes a big difference.

PS: I may take a while to reply back.

Yes, opcache is active.
Settngs from phpinfo:
opcache.blacklist_filename => no value => no value
opcache.consistency_checks => 0 => 0
opcache.dups_fix => Off => Off
opcache.enable => On => On
opcache.enable_cli => Off => Off
opcache.enable_file_override => Off => Off
opcache.error_log => no value => no value
opcache.file_cache => no value => no value
opcache.file_cache_consistency_checks => 1 => 1
opcache.file_cache_only => 0 => 0
opcache.file_update_protection => 2 => 2
opcache.force_restart_timeout => 180 => 180
opcache.huge_code_pages => Off => Off
opcache.interned_strings_buffer => 8 => 8
opcache.lockfile_path => /tmp => /tmp
opcache.log_verbosity_level => 1 => 1
opcache.max_accelerated_files => 10000 => 10000
opcache.max_file_size => 0 => 0
opcache.max_wasted_percentage => 5 => 5
opcache.memory_consumption => 128 => 128
opcache.opt_debug_level => 0 => 0
opcache.optimization_level => 0x7FFEBFFF => 0x7FFEBFFF
opcache.preferred_memory_model => no value => no value
opcache.preload => no value => no value
opcache.preload_user => no value => no value
opcache.protect_memory => 0 => 0
opcache.restrict_api => no value => no value
opcache.revalidate_freq => 2 => 2
opcache.revalidate_path => Off => Off
opcache.save_comments => 1 => 1
opcache.use_cwd => On => On
opcache.validate_permission => Off => Off
opcache.validate_root => Off => Off
opcache.validate_timestamps => On => On

Hi @clemente.raposo
The network tab shows some 10 lines. It takes some 6 secs to first action, then at mark 40 secs the list starts to show. At mark 80 secs the list is complete , but the page “unusable”, but not stone dead. It does not recover to some descent status.
The DB call times according to Workbench are roughly the same for Linux box calls as from Andriod.

Update:
Testing today towards an almost empty DB. We imported some old Accounts data. 73 records.
Running site from Safari browser the access is rather quick. Have no figures.
Running the same with Lunux box and Firefox 96.0 it is very slow. Some 40+ secs until any data present.
Now , my theory is that the problems lies on client side in some configurations.
I will by glad for any hint pointing to a possible cause.
Possible routes for us:

  1. Manual input of data in version 8 instance.
  2. Finding the culprit
  3. Running something else
  4. Version 7.

Experience report.
The signs of resource problem causing a slow display of just one list (Accounts) didn’t occur in all platforms. Safari in Mac and access from Android went fine. It led us in wrong direction at first. The main culprit was to low settings in opcache, though increase from default. Some are now to big for server used so we have a descission what to do. Probably we increase the server memory and keep opcache as is. CPU load in not a problem. Mysql process using some 55% memory but very short periods. Upgrade server also will help that process.
Now and then the swapping kicks in and everything running veeeery sloooow.

And, yes, the move and migration of version 6 data to version 8 went fine, so far.

Thanks for your thought, advices and support

2 Likes

Hi @Kenta,

Sorry for the delay in replying.

Thank you for sharing!