SuiteCRM memory requirements?

pgr · 19 February 2020 15:20

there are known issues with the upgrades, they have been solved but some of them will only bring effects in the next upgrades…

See the release notes

and this fix (just to learn - it is already part of the newest packages)

github.com/salesagility/SuiteCRM

Interrupted / Hanging Upgrades

opened 11:20AM - 15 Nov 19 UTC

closed 01:03PM - 25 Mar 20 UTC

pgorod

Type: Bug Area: Upgrading

#### Issue Several manifestations of Upgrade problems. The most frequent are th…e upgrade hanging at a certain step, or at the very end (in which case the upgrade actually succeeds in taking you to the next version). Sometimes there is a HTTP 500 error sent to the client. Sometimes there is a `php zend_mm_heap corrupted` in the Apache log. The problem occurs due to long delays in cleaning up the temp directory, which crash PHP in a way that doesn't leave any traces in the logs most of the times... the problem tends to aggravate significantly after occurring more than once, since the files accumulate in this directory. #### Quick Workaround If you just want to get your upgrade done, often all that is needed is to clear the entire contents of `cache/upgrades/temp` and start again. #### Actual Behavior The [function `unlinkUWTempFiles`](https://github.com/salesagility/SuiteCRM/blob/v7.11.10/modules/UpgradeWizard/uw_utils.php#L2258) calls a [function `findAllFiles`](https://github.com/salesagility/SuiteCRM/blob/v7.11.10/include/dir_inc.php#L206) which is _very_ inefficient for large directories. It recursively goes through every file in the directory tree, adding each element to an array, then sorting it, then sorting it again, and then deleting each file individually. So it's a long period of disk read activity, starving the process memory, followed by a long period of CPU intensive operations, followed by a long period of disk write activity. #### Possible Fix This sort of directory iteration should be done using a PHP language construct called `Iterator`. Instead of making a huge array of files first, and then looping through each item to perform some function, it should just get the necessary item at the moment it needs to act on it. No significant memory consumption, and much simpler code. I will provide a PR. #### Steps to Reproduce 1. Run an upgrade to one of the newest versions, like 7.10.9 or 7.10.10. 2. Keep an eye on `cache/upgrads/temp` to see how many copies of the unzipped package you have there. They add up after each time the process hangs. 3. If necessary to help reproduce, make these copies yourself. Fill that directory with tens of thousands of files. Each package has around 22k. I had a real-life situation with over 90k files. 4. After the upgrades conclude, this directory should be completely empty. If not, then this issue occurred. #### Additional tools that might help (Linux, needs `sudo`) * `htop` - a fancier `top`. Shows CPU usage and other details. when you see an Apache process that is taking up 100% of the CPU, note its `PID`. * `iotop` see aggregated disk activity, by process. Similar to `top` or `htop`, but for disk access. * `lsof -n -p 1234 | grep -v mem` - get a momentary glimpse of what a process with PID `1234` is doing in terms of file accesses, file by file. You can repeat this command and see that it's iterating the directory, and check if it's reading or writing. * `watch` - repeats a command every `n` seconds. Useful to leave separate terminals showing you regular updates of the above tools' output. I used these ```bash Observe file accesses of process with PID 1234: watch -n 2 'lsof -n -p 1234 | grep -v mem' Track directory size dynamically: watch -n 2 'du -h --max-depth=1 cache/upgrades/temp' ``` #### Environment * SuiteCRM Version used: problems started around version 7.10.7 up to 7.10.10