New Marketing Record on old Campaign results in suppressions

Understood what has been said here:

The Email Marketing record is the actual scheduling of the group of emails using a defined template. An Email Marketing record is associated to the template and this is associated with the campaign. So for example, an Email Campaign will use a single Email Marketing record with a single Template. However a Newsletter Campaign (as it will more likely be a rescheduled campaign with repeat emails but at different times) you can create multiple differen t scheduled group of emails (each with their own template or a reselect the previous template) to send out and all the tracker info is recorded on the same campaign.

But it doesnā€™t work here. Using an existing Campaign which has already launched emails to a Target-List, lauching the wizard, on Marketing tab, when I change the Marketing Email Name, adjust schedule and Next to Send Email and Summary and then push Send Email at Scheduled Time, the emails, I can see the emails queued and, when I push Send Queued Campaign Emails, they are all Suppressed by Address or Domain if the Target-List was not changed.

The queue scheduler also doesnā€™t work, although cron is running and nightly email scheduler log has all the right entries, but thatā€™s another story, I can live with the ā€œSendā€ button.

Can someone help, please?

Generally speaking, this is a very good system, congratulations and thank you for letting me use it. Iā€™ll try to find a way to introduce it to my customer base, but I must sort out a few details first.

Strange. But itā€™s possible that something is not quite the way I was seeing it. Itā€™s a complex part of SuiteCRM, and not always very well developed.

Are you a PHP developer? If so, I could point you to the relevant parts of the code, to help you explore a bit deeper.

I can help you fix the cron problems if you give more info. You can start with this, it should give you all the help you need:

1 Like

Hello. Thank you for helping.

Iā€™m not a PHP developer but I can reasonably understand what the code is doing. Iā€™m mostly a ā€˜Cā€™ developer although I use to think that languages are only a manual away; that is, if you are not too ambitious about the length and quality of the code you will produce. My usual environment is *nixes.

That said, my data user is ā€˜wwwā€™ which is allowed_cron_user in config.php, my server OS is Freenas (FreeBSD), the webserver is running in a jail and my www crontab is:

root@webserver:/ # sudo -u www crontab -l
*/15 * * * * /usr/local/bin/php -f /usr/local/www/apache24/data/nextcloud/cron.php
0 2 * * * /usr/local/www/apache24/data/nextcloud/occ preview:pre-generate
* * * * * cd /usr/local/www/apache24/data/crm; php -f cron.php > /dev/null 2>&1

I guess everything the last line is correct, isnā€™t it?

One thing I didnā€™t say is that, on the campaign status view, on the Message Sent/Attempted panel and Viewed Message panel, the Marketing ID column changes to the value of the last Marketing ID I issued, even on records already existing that had previous Marketing IDā€™s, which shouldnā€™t happen. I hope my explanation was clear enough.

Thank you again for your time and support.

When you go in Admin / Schedulers, the instructions at the bottom, what do they say?

(just checking that your web server is running under user www, as seen from inside the app)

Thank you again.

In order to run SuiteCRM Schedulers, edit your web server user's crontab file with this command:
**sudo crontab -e -u www**
... and add the following line to the crontab file:
***    *    *    *    *     cd /usr/local/www/apache24/data/crm; php -f cron.php > /dev/null 2>&1**

By the way, job log:
Run Nightly Process Bounced Campaign Emails done 2020-10-19 10:30 2020-10-19 10:30
Run Nightly Process Bounced Campaign Emails done 2020-10-19 10:25 2020-10-19 10:25
Run Nightly Process Bounced Campaign Emails done 2020-10-19 10:20 2020-10-19 10:20
Run Nightly Process Bounced Campaign Emails done 2020-10-19 10:15 2020-10-19 10:15
ā€¦

This tells us that scheduler Nightly Process Bounced Campaign Emails is running, doesnā€™t it?

The user seems correct and yes, the job appears to be running.

If you go in Admin / Schedulers and enter the specific job, does it show a Last ran successfully that is recent and in the correct timezone?

Yes, just a while ago, at 12:53:
Run Nightly Process Bounced Campaign Emails done 2020-10-19 12:50 2020-10-19 12:50

I have it set to run every 5 minutes.

Oops, thatā€™s not it, not bounced, and the timezone doesnā€™t seem correct, my bad:

Run Nightly Mass Email Campaigns done 2020-10-19 06:00 2020-10-19 06:00
Run Nightly Mass Email Campaigns done 2020-10-19 05:00 2020-10-19 05:00
Run Nightly Mass Email Campaigns done 2020-10-19 04:00 2020-10-19 04:00

Iā€™ll investigate it deeper and let you know.

Only the run interval was wrong, timezone is correct. But this probably explains why the queue was not processed at 09:00 as I wanted:

Run Nightly Mass Email Campaigns	done	2020-10-19 13:00	2020-10-19 13:00	
Run Nightly Mass Email Campaigns	done	2020-10-19 06:00	2020-10-19 06:00	
Run Nightly Mass Email Campaigns	done	2020-10-19 05:00	2020-10-19 05:00

I donā€™t think this could affect the suppressions, my main problem, but if you donā€™t agree, I can load the email queue with another Marketing ID and test.

The timezone for that needs to be set in the php.ini of the CLI.

Check it with

php -i | grep timezone

Change it from the file given by

php -i | grep php.ini

Then restart web server.

I am unsure about the Email queuing part, so itā€™s probably a good strategy to try and get this working accurately, then restart all your testing from scratch (new campaign).

Right:

root@webserver:/ # php -i | grep timezone
Default timezone => Europe/Lisbon
date.timezone => Europe/Lisbon => Europe/Lisbon
root@webserver:/ # php -i | grep php.ini
Configuration File (php.ini) Path => /usr/local/etc
Loaded Configuration File => /usr/local/etc/php.ini
root@webserver:/ # cat /usr/local/etc/php.ini | grep timezone
; Defines the default timezone used by the date functions
; http://php.net/date.timezone
date.timezone = Europe/Lisbon

I guess this is correct, no need to reboot, right?

Yep that sounds correct

Sorry to ask, any other guidance, please?

Ahh right. So: your C skills are just fine. I also learned C/C++/C# and went into PHP without a second thought.

The Queuing takes place here, when you save a Campaign:

modules/Campaigns/QueueCampaign.php

I am not sure when it does when re-queuing - I would expect itā€™s when you change the marketing record. From this point on, you can almost forget about the concept of campaign. Itā€™s just a list of email addresses in emailman database table that will get selected and sent (after testing a bunch of conditions for dates, suppressions, etc).

Then the actual sending takes place here:

modules/EmailMan/EmailManDelivery.php

See suppressions-related code starting around line 187. For individual suppressions focus on the several calls to the set_as_sent function. These suppressions depend on the data in campaign_log database table.

Thank you, I have plenty to dive in.

In the meanwhile, doesnā€™t it ring a bell to you that existing emails, already sent, get their marketing ID changed?

If the comparison occurs after this, of course there must be coinciding emails, meaning, same adress because they were already sent, and same Marketing ID because it was just changed to the new one. Does this make any sense to you?

I wouldnā€™t be surprised if there was a bug (or bugs) lurking here.

I would also like to understand better this whole marketing_id thing. Iā€™ve studied this Campaigns code a lot for the work I did on my PowerReplacer add-on, adding full Previews to Campaigns. I understand it a lot better than I did before, but still not completely. Itā€™s very messy and opaque.

I would like to one day completely redesign this campaigns wizard. Everything needs to be so much clearer when youā€™re about to start hitting customers with tons of email. You really shouldnā€™t have to hesitate about whoā€™s going to receive what.

I donā€™t really have a specific answer to your question (Iā€™m hoping you can find it :slight_smile: ). I havenā€™t seen any code writing new marketing_ids to emails tables; but if you can confirm itā€™s happening we can always go find itā€¦

Well, campaigns are one of the most useful functions of any crm, as Iā€™m sure youā€™ll agree. But, as you said, we must be certain of who receives what and we must be able to trust this. So, youā€™re right, it deserves a redesign.

Concerning the marketing_ids, as I said before, I can see on the campaign status window, on the message sent/attempted panel that the marketing_ids of existing mesages, whenever I issue an old campaign with a new value of marketing_id, change to this new value, that is, all the list has now the last marketing_id value. This also happens on the message viewed panel.

I would find strange that the values arenā€™t changed on the db table column, but only on the view.

So, Iā€™m planning for later today to set up my terminal and prove that the field really changes, and post the results here for your information.

1 Like

All right, I understand now what is happening, but not where in the code it is originated.

So, when I launch the wizard of an existing campaign and proceed up to the Marketing tab, I now have to edit the field Marketing Email Name to supposedly create a new marketing record.

But this is not happening. In fact, on the table email_marketing, instead of a new record being created, the existing one is being edited, changing the column ā€˜nameā€™ the the text I had entered above. The Id field, which is the index, of course remains the same and so the Marketing record is effectively the same, with a changed ā€˜nameā€™.

Thatā€™s why the messages are being suppressed, that is, same email address and same Marketing Id.

Furthermore, this explains why previously existing messages, on the campaign status panel, on the log of sent/attempted messages, the column Marketing Id changes to the new ā€˜nameā€™.

And also why on the Campaign Wizard, Send Email and Summary tab, Select Email Marketing Record drop down, thereā€™s only one option to choose from, exactly the last one inserted.

Now, the question is: where in the code is the email_marketing table being edited instead of a new record created?

Thank, weā€™re getting closer! This is interesting.

Thereā€™s a secret deprecated screen I sometimes use:

First, go in modules/EmailMarketing/EmailMarketing.php and change the initial if elseif block to become just this:

if (isset($_REQUEST['record']) && $_REQUEST['record']) {
    $focus->retrieve($_REQUEST['record']);
}

Now visit a URL like this:
http://localhost:8080/index.php?module=EmailMarketing&action=EditView&record=b0566b05-91a0-bd00-e5c7-5eb1484f1e7c

Replace that with your domain name, and Marketing id from your system.

You should see an edit view for MrketingRecords, without getting redirected to the wizard.

Does that help?


About the code that currently changes the name instead of the id, have a look at

modules/Campaigns/WizardMarketingSave.php
modules/Campaigns/WizardNewsletterSave.php
modules/Campaigns/WizardCampaignSave.php

one of these should be responsibleā€¦

Thank you. It doesnā€™t work with the modified if elseif, just a blank window, but it does work with the original file, I can see the whole record with a small window on the bottom containing the remainder records. Didnā€™t try to save to check whether it goes to the wizard.

Anyway, I donā€™t need the editor window, I use Squirrel database manager for that.

Concerning the potential culprits, Iā€™ve checked them briefly but I couldnā€™t find the place where the record is being edited instead of created. Iā€™ll keep trying.