Bulk deleting records timesout page and returns 503 error

💬 General Discussion
1

I’ve set our CRM to show 100 records per page and when I bulk select all the Cases I want to delete, the page eventually times out.

I’ve tried this many times over and it never works. Is there a way to debug this problem?

Many thanks

2

I have since changed this to 20 and still the same thing happens

3

Changed to 10 now but still the same problem

4

If I tick just one Case, then delete, all works well

5

This comes from the system error log. Not sure what I should be looking out for?

Stack trace:
#0 /usr/www/users/1234jgmtc/help.1234-98765.com/lib/Search/ElasticSearch/ElasticSearchHooks.php(105): SuiteCRM\Search\ElasticSearch\ElasticSearchHooks->reIndex(Object(aCase))
#1 /usr/www/users/1234jgmtc/help.1234-98765.com/lib/Search/ElasticSearch/ElasticSearchHooks.php(92): SuiteCRM\Search\ElasticSearch\ElasticSearchHooks->reIndexSafe(Object(aCase))
#2 /usr/www/users/1234jgmtc/help.1234-98765.com/include/utils/LogicHook.php(272): SuiteCRM\Search\ElasticSearch\ElasticSearchHooks->beanDeleted(Object(aCase), ‘after_delete’, Array)
#3 /usr/www/users/1234jgmtc/help.1234-98765.com/include/utils/LogicHook.php(213): LogicHook->process_hooks(Array, ‘after_delete’, Array)
#4 /usr/www/users/1234jgmtc/help.1234-98765.com/data/SugarBean.php(3112): LogicHook->call_custom_logic(‘Cases’, ‘after_delete’, Array)
#5 /usr/www/users/1234jgmtc/help.1234-98765.com/data/SugarBean.php(5244): SugarBean->call_custom_logic(‘after_delete’, Array)
#6 /usr/www/users/1234jgmtc/help.1234-98765.com/include/MassUpdate.php(286): SugarBean->mark_deleted(‘7a1233d9-b04b-6…’)
#7 /usr/www/users/1234jgmtc/help.1234-98765.com/include/MVC/Controller/SugarController.php(781): MassUpdate->handleMassUpdate()
#8 /usr/www/users/1234jgmtc/help.1234-98765.com/include/MVC/Controller/SugarController.php(525): SugarController->action_massupdate()
#9 /usr/www/users/1234jgmtc/help.1234-98765.com/include/MVC/Controller/SugarController.php(494): SugarController->do_action()
#10 /usr/www/users/1234jgmtc/help.1234-98765.com/include/MVC/Controller/SugarController.php(468): SugarController->handle_action()
#11 /usr/www/users/1234jgmtc/help.1234-98765.com/include/MVC/Controller/SugarController.php(373): SugarController->process()
#12 /usr/www/users/1234jgmtc/help.1234-98765.com/include/MVC/SugarApplication.php(113): SugarController->execute()
#13 /usr/www/users/1234jgmtc/help.1234-98765.com/index.php(54): SugarApplication->execute()
#14 {main}

6

Some general suggestions:

  1. If you have too much data in database an too much to delete(Cases linked to Accounts etc), make sure that Scheduler has ran with Prune Database. And then make sure that your tables optimized/repaired in your DB. You can do that via phpmyadmin for MySQL.

  2. Make sure that Schedulers and scheduled tasks are not running during your delete. Sometimes they interfere with delete actions.

  3. Make sure that php parameters adjusted for long running tasks.

  4. Make sure that httpd timeout parameters adjusted for long running tasks.

  5. Check the server for running processes after you get 503 on your screen, sometimes backend continues on delete while the frontend times out.

7

Thanks @obekatli

Couple of questions?

3. Make sure that php parameters adjusted for long running tasks.
Where would I do this?

4. Make sure that httpd timeout parameters adjusted for long running tasks.
Where is this done

5. Check the server for running processes after you get 503 on your screen, sometimes backend continues on delete while the frontend times out.
I am on a dedicated managed server and don’t have access to this. Is there a way to check server logs?

8

I have managed to increase the server timeout and that seems to have deleted the first lot and not timing out but it does still take an incredibly long time

9

  1. php parameters are usually set in php.ini file in /etc. You need to review max_execution_time and
    max_input_time parameters in php.ini file.

  2. If you use Apache httpd as your webserver, you need to review TimeOut and ProxyTimeout parameters in conf file for your SuiteCRM VirtualHost

  3. If you have shell access you list running processes and check if any long running php process exists after your timeout.

Remember that, SuiteCRM does not automatically physically delete rows from DB tables, instead it marks as deleted. Only the Prune Database scheduled task deletes them physically(unless you delete the records with deleted=1 directly from DB). Therefore the deleted Cases will not visible on front-end but do exist in DB. If your DB is large(so many Accounts, Cases etc) would be better to Prune DB then optimize DB then re-try for delete.

10

Yes, because of the logic DB tables organised and processed by SuiteCRM, it takes too much time even you performed Prune/Optimize. There are some guidelines/suggestions for achieving better performance for bulk actions, if you do them more frequently.

And if you are going to delete thousands of cases, it will time out like, your screen become free(without any timeout message) you can either continue work on same browser session are you need to close your browser(with all tabs) and re-launch and re-login and see some of them deleted already and when you check after some minutes, you will see some more Cases deleted and it goes after finishing marking all as deleted.

Make sure that your Schedulers run from crontab regularly and perform DB optimise as required. For smaller DB sizes it is not required even to mention about. But after deleting so many records, it would be beneficial to run all pre-defined, SuiteCRM supplied scheduled tasks.