Very Large aow_processed and aow_processed_aow_actions tables.

pgr · 3 October 2018 17:20

Fixed the typo, thanks.

From what I see in the code (just a quick look):

https://github.com/salesagility/SuiteCRM/blob/hotfix/modules/AOW_WorkFlow/AOW_WorkFlow.php (search for “modified”)

… it seems it always compares the records date_modified with their date_entered values, regardless of running from “on save” or “on scheduler”. So I would say it still applies as a restriction.

But I am not sure what date_entered means when “on scheduler” :huh:

If you have some time to test or run it with a debugger, I’d love to get to the bottom of this.

simonno · 5 October 2018 14:10

Hello I’m running into the problem that both of my tables are bigger than 10GB, making 20GB together. I tried the sql-query to delete some lines but the execution always fails (as the tables might be to big). Do you have an advice on what to do now?

Best regards

Simon

pgr · 5 October 2018 14:32

Maybe try making the Query more restrictive, doing it by chunks (year by year, or even month by month).

John_EZS · 5 October 2018 14:58

did you change the date filters from 2017-10-15 00:00:00 to something more recent, like maybe 2018-09-01 00:00:00?

Even if your tables are huge, the select statement should work fine I would think?

Also, at least for debugging, don’t run the whole script in one go. Instead run the individual sql queries one at a time. Then at least you’ll know which queries aren’t working.

simonno · 5 October 2018 16:04

I’m going to try that. Thank your for the advice!

simonno · 6 October 2018 05:45

Hello John and pgr,

i set a more recent date and split the whole script and after waiting a couple of hours, i was finally able to go through the whole script.

The aow_processed table still contains 3 Mio. lines now, so this one is getting bigger and bigger, month by month. The other table is ok. Would be great to see a solution to this whole table problem.

Thanks again for your help!

John_EZS · 6 October 2018 12:49

I agree! Please upvote this post on github.

We would like to see this or something better be implemented as standard database maintenance behavior.

holdusback · 4 March 2019 15:35

I’ve put this topic in my fav, cause for now Im in “test” mode, I’ve got like 10 account 30 contact ect … I’ve dont import my database yet, but thoses tables are already the biggest I’ve got :aow_processed, job_queue and aow_processed_aow_actions.

They are full of logs from the beggining, wich I dont really care about. I’ve got the scheduler “PRUNE DATABASE ON 1ST OF MONTH” running, in the log I got the last execution:

Prune Database on 1st of Month done 01/03/2019 04:00 01/03/2019 04:00

But I still got a lot of data, for exemple in job_queue I got data from the 21/02.

For now Im deleting thoses table when I can, but when I’ll import my real database it will be much larger since I’ve got like 3k5 acc, full of contact and Opportunities.

Do someone got a solution like a CRON or a maybe a view in database ?