Job runs too frequently, throttled to protect the system.

👨‍💻 Development Help
1

Hey there, again a topic about this common issue : Job runs too frequently, throttled to protect the system.

Got this at random time, not every hour, just random time.

I’ve inspected the log in debug mode, I have 3 jobs that are launched every hour and im nearly sure the issue come from : GOOGLE MAPS GEOCODING PROCESS

Here are the two lines that tell me it come from here :

Tue Sep 24 15:00:04 2019 [30611][1][DEBUG] ----->Firing was successful: https://xxxx.com/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1
Tue Sep 24 15:00:04 2019 [30611][1][DEBUG] ----->WTIH RESULT: HTTP/1.1 301 Moved Permanently

In the scheduler i have this in URL : url::https://xxxx.com/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1

So maybe its an encode issue ?

The strange thing is that in the log it says that it’s a success.

Tue Sep 24 15:00:04 2019 [30611][1][INFO] Query:UPDATE job_queue
					SET `assigned_user_id`='1',`name`='Google Maps Geocoding Process',`date_modified`='2019-09-24 13:00:03',`scheduler_id`='648ef1e9-704f-96c9-e789-5d02083b48e2',`execute_time`='2019-09-24 13:00:00',`status`='done',`resolution`='success',`message`=NULL,`target`='url::https://xxxx.com/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1',`data`=NULL,`requeue`=0,`retry_count`=NULL,`failure_count`=NULL,`job_delay`=0,`client`='CRONa0af2a63b6342f94d4a6e68c780ac405:30611',`percent_complete`=NULL
					 WHERE  job_queue.id = 'bbda6ba9-f9be-67fa-36ff-5d8a13f5d14f' AND deleted=0

Ofc, the geocoding not work. But I found that during the debug of my main issue (Job runs too frequently) so I assume that I have some time the runs too frequently because of that.

I have tried to turn of “re launch if missed” but I still have the error log.

Hope someone can help, I’ve been doing detective thing in log for a week now, without founding a proper fix !

2

I’ve checked in the database, it save correctly the URL :

(‘648ef1e9-704f-96c9-e789-5d02083b48e2’, 0, ‘2019-06-13 08:25:45’, ‘2019-09-06 09:30:39’, ‘1’, ‘1’, ‘Google Maps Geocoding Process’, ‘url::https://xxxx.com/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1’, ‘2005-01-01 09:30:00’, NULL, ‘0::/1::::::’, NULL, NULL, ‘2019-09-16 08:00:01’, ‘Active’, 1),

So its during the process that it change the URL.

And I can add that it redirect to the login page after the 301 error :

“GET /index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1 HTTP/1.1” 301 - xxxx.com “-” “-” “-”
“GET /index.php?action=Login&module=Users&login_module=jjwg_Maps&login_action=index HTTP/1.1” 200 14404 xxxx.com “-” “-” “-”

3

So, I think I finally found my issue, the thing with this small error-log, is that it’s not specific to a job. So it will fix MY issue, but not all…
I think this error log really need to mention the job, this way we will skip that “detective” part.

I fixed the 301 error that I mention in my last post and now I don’t have the error anymore.

Before my fix I had this : /index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1
But in the database its correctly saved so its during the process :

In /SuiteCRM/modules/SchedulersJobs/SchedulersJob.php

Line 555

        
} elseif ($exJob[0] == 'url') {
            if (function_exists('curl_init')) {
                $GLOBALS['log']->debug('----->SchedulersJob firing URL job: '.$exJob[1]);
                set_error_handler(array($this, "errorHandler"), E_ALL & ~E_NOTICE & ~E_STRICT);
		// Ici original remplacer par le mien 
                // if ($this->fireUrl($exJob[1])) {
                if ($this->fireUrl(html_entity_decode(html_entity_decode($exJob[1])))) {
                    restore_error_handler();
                    $this->resolveJob(self::JOB_SUCCESS);
                    return true;
                } else {
                    restore_error_handler();
                    $this->resolveJob(self::JOB_FAILURE);
                    return false;
                }

I just change : if ($this->fireUrl($exJob[1])) {

To : if ($this->fireUrl(html_entity_decode(html_entity_decode($exJob[1])))) {

I use 2 time html_entity_decode() cause it seem it has been encoded 2 times, so it fires correctly my URL without thoses ;amp thing.

In my server log it confirm it :
[30/Sep/2019:04:00:01 +0200] “GET /index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1 HTTP/1.1” 200 - XXXX.com “-” “-” “-”

So @PGR its maybe cause of my setting that it encode it this way ? Or it is a common error for everyone but someone would saw it before me in this case… I think I need to open a github issue on this

1 Like
4

Keep an eye on this issue, it seems to be related

https://github.com/salesagility/SuiteCRM/issues/7704

1 Like
5

THANK YOU!

I cannot believe that in Oct 2022 this bug still exists, but it does.

Your fix let me get it working again.

6

We updated to SuiteCRM 7.14 and PHP7.4.

We have a geocode cron job which set in scheduler.

./index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1

But it is failing, getting below message on screen.

The job failed. Check the logs to see went wrong.

We checked server error log and php error log, but we don’t have any error there. We have the following line on sugarcrm log file.

Job failed: http://NAME/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1

We don’t know, why it stopped working. Any suggestions and correct directions are appreciated.

Thank you!

7

@rsp

I had issues with the geocode setup as well.

Did you check the bug fix posted above by @pgr? It was still required in my system.

Google requires https not http

Also, the required Job URL was the full URL, not just the part after the domain

In the SuiteCRM documentation, it says to enter “./index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1”

in the URL field. However, this did not work and in the forums, the URL to enter was said to require the full URL - which did work

[https://{URL_of_SuiteCRM_site}/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1]

I am assuming you actually have your domain instead of NAME in your URL. Make sure it is the full path to your SuiteCRM system (ie if in a subdirectory, include the subdirectory)

1 Like
8

Thank you so much :smile: @Ramblin

Found issue!

Job URL was wrong in the scheduler. We corrected it and now it is showing “The job completed successfully.” in the Kickoff Job.

I hope this will help someone in the future.

9

@Ramblin.

We are running a crm system on http://ip_address/ for now.

We put entire URL for geocode cronjob but it is failing now. I tried it few days back, it was running successfully.

Do I have to make any changes? I did not understand that bug fix on GitHub.

I am using below link for job to run.

http://<IP_ADDRESS>/<CRM_NAME>/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1

Thanks in an advance!

10

If your system is at the root directory of your web server, you do not want to have the /<CRM_NAME> subdirectory included in the URL. If your syste is in a subdirectory of the web server directory, then yes, include the subdirectory name where you have /<CRM_NAME>

As for the bug, see below.

THe other thing that can interfere with geocoding is if you use basic authentication to secure access to your server. If so, I show below what to do about that as well.

================================================

There is a bug in SuiteCRM which has been around for quite some time and therefore is not expected to be fixed in v7 now that SuiteCRM has moved to v8, so the fix is documented here. There is a encode/decode issue when SuiteCRM takes URLs from the Scheduler and tries to actually take the desired action. This fix is NOT upgrade safe so after an upgrade that changes the file shown you will have to re-do this fix.
In modules/SchedulersJobs/SchedulersJob.php on around line 550
nano --linenumbers modules/SchedulersJobs/SchedulersJob.php

Change

if ($this->fireUrl($exJob[1])) {

To

if ($this->fireUrl(html_entity_decode(html_entity_decode($exJob[1])))) {

Rebuild and Repair

When combined with the correction below for what to put in the URL field, the automatic geocode lookups will now work.

Once we enable directory protection with Basic Authentication with Apache, the Google geocoding APIs stopped working. So we needed to add the Google mapping/geolocation URLs to the hosts that could access our system.
At the google page List of Google Maps Domains  |  Google Maps Platform  |  Google for Developers , Google has a LONG list of the URLs it uses for various functionality with its APIs. Listed below are the URLs required to fully enable the geolocation scheduler for SuiteCRM. Add those hosts to the Apache Basic Authentication setup at the top of the .htaccess file in {SuiteCRM Directory}. Make sure to include the CNAMEs and do not just have the top-level domain defined; the CNAMEs actuall generate a different list of IP addresses. The section now looks like
# Create directory-level authentication
# as an extra layer of protection against hackers
# exempt mydomain.com so it can be used (in the future) for web form leads
# exempt mydomain.com so its subdomains can be used for trackers and images
AuthType Basic
AuthName “First-Level Authentication”
AuthUserFile /home/customer/www/security/.htpasswd
Require forward-dns mydomain.com
Require forward-dns www.googleapis.com
Require forward-dns maps.googleapis.com
Require forward-dns maps.gstatic.com
Require forward-dns fonts.gstatic.com
Require valid-user

The forward-dns option requires Apache v 2.4.19 or higher. The historical “Require host …” will not work since the Google URLs are associated with many IP addresses (do an nslookup on google.com to see) and the "Require host … " only worked with hosts that had one IP address associated with it. the “Require forward-dns …” works with URLs that have multiple IPs associated with it. To check the version of Apache running on your server:

11

It is weird that my another instance is running with ip_address/crm_name/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1

I will dig down more and post my update.

crm log error is

Job failed: http://xx.xx.xx.xx/xxxxxx/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1

12

Geocode cron job is now running successfully. We didn’t change URL.

Used URL:

url::http://xx.xx.xx.xx/xxxxxx/index.php?module=jjwg_Maps&entryPoint=jjwg_Maps&cron=1