Grep'ing logs for most common messages

Hi everybody,

I just solved a problem for myself and I though this one would be nice to share here.

I wanted to have a look at my logs (suitecrm.log and php_errors.log) and determine which are the most common messages in there.

So I came up with these (use from your root directory):

grep '' suitecrm.log | cut -d']' -f3- | sort | uniq -c | sort -nr | head -n10

That will give you the top-ten of all most common messages, with a nice little count of occurrences. If you want to narrow it down by kind of message (info, debug, warning, error, fatal) just use this:

grep 'WARNING' suitecrm.log | cut -d']' -f3- | sort | uniq -c | sort -nr | head -n10

For php_errors it’s a little different:

grep ': ' php_errors.log | cut -d' ' -f4- | sort | uniq -c | sort -nr | head -n10

And for the narrow version, use this for example:

grep 'Error: ' php_errors.log | cut -d' ' -f4- | sort | uniq -c | sort -nr | head -n10

Now I’m off to determine why one of my hooks that is supposed to run when a contact is saved, is being called 4 times a second on an idle test system… : - )

1 Like

And in case anyone wants to compare, here is the output for suitecrm.log


   1050 [INFO] Query:SELECT lhs_module, rhs_module FROM relationships WHERE deleted=0 AND (lhs_module = 'SecurityGroups' OR rhs_module='SecurityGroups')
   1050 [DEBUG] SecuritySuite: Get SecuritySuite Enabled Modules: SELECT lhs_module, rhs_module FROM relationships WHERE deleted=0 AND (lhs_module = 'SecurityGroups' OR rhs_module='SecurityGroups')
    638 [DEBUG] ----->Schedulers->deriveDBDateTimes() got an object of type: Scheduler
    638 [DEBUG] Hook called: Schedulers::process_record
    638 [DEBUG] ----->got * months
    638 [DEBUG] ----->got * day
    526 [DEPRECATED] SugarBean.php: preprocess_fields_on_save() is deprecated
    526 [DEBUG] Creating new instance of hook class AssignGroups without parameters
    526 [DEBUG] Creating new instance of hook class AOW_WorkFlow without parameters
    526 [DEBUG] Creating new instance of hook class AOD_LogicHooks without parameters
    525 [INFO] Query:SELECT id FROM aow_workflow WHERE aow_workflow.flow_module = 'SchedulersJobs' AND aow_workflow.status = 'Active' AND (aow_workflow.run_when = 'Always' OR aow_workflow.run_when = 'On_Save' OR aow_workflow.run_when = 'Create') AND aow_workflow.deleted = 0
    525 [DEBUG] Hook called: SchedulersJobs::before_save
    525 [DEBUG] Hook called: SchedulersJobs::after_save
    464 [DEBUG] ----->got * dates
    350 [INFO] Query:UPDATE job_queue
    349 [INFO] Query:SELECT users.* FROM users  WHERE users.id = '1' AND users.deleted=0 LIMIT 0,1
    349 [INFO] Query:SELECT u1.first_name, u1.last_name from users  u1, users  u2 where u1.id = u2.reports_to_id AND u2.id = '1' and u1.deleted=0
    349 [INFO] Query:SELECT ea.email_address, ea.email_address_caps, ea.invalid_email, ea.opt_out, ea.date_created, ea.date_modified,

Some of those are part of blocks and always happen together. They’re easy to spot because the occurrence counts match.

And for php_errors.log:

    454 PHP Warning:  Illegal string offset 'cookie_name' in /var/www/html/cache/smarty/templates_c/%%E2^E2B^E2BC33C5%%SubPanelTiles.tpl.php on line 41
    442 PHP Warning:  Declaration of SugarEmailAddress::save($id, $module, $new_addrs = Array, $primary = '', $replyTo = '', $invalid = '', $optOut = '', $in_workflow = false) should be compatible with SugarBean::save($check_notify = false) in /var/www/html/include/SugarEmailAddress/SugarEmailAddress.php on line 0
    424 PHP Notice:  Undefined variable: color in /var/www/html/custom/modules/Contacts/ContactsPgrProcessHook.php on line 28
    288 PHP Warning:  date() expects parameter 2 to be integer, string given in /var/www/html/custom/modules/Contacts/ContactsPgrProcessHook.php on line 30
    140 PHP Notice:  A non well formed numeric value encountered in /var/www/html/custom/modules/Contacts/ContactsPgrProcessHook.php on line 30
     82 PHP Notice:  Trying to get property of non-object in /var/www/html/include/Dashlets/Dashlet.php on line 245
     58 PHP Notice:  Undefined index: editor in /var/www/html/include/SugarFields/Fields/Text/SugarFieldText.php on line 89
     45 PHP Warning:  Declaration of SugarFeedDashlet::process($lvsParams = Array) should be compatible with DashletGeneric::process($lvsParams = Array, $id = NULL) in /var/www/html/modules/SugarFeed/Dashlets/SugarFeedDashlet/SugarFeedDashlet.php on line 0
     45 PHP Notice:  Undefined property: JotPadDashlet::$seedBean in /var/www/html/include/Dashlets/Dashlet.php on line 245
     42 PHP Notice:  Undefined index: LBL_UPDATE_TEXT in /var/www/html/modules/AOP_Case_Updates/Case_Updates.php on line 351

Have fun!