Campaign Email Marketing record set to "Sent" before run date – emails stay in queue forever when scheduled in advance

📦 Install & Upgrade Help
,
1

Issue

When an Email Marketing record is scheduled for a future date, the emails are queued before the scheduled run date. If the “Send Campaign Emails” scheduler runs after queueing but before the run date, it finds no due entries and sets the record to “Sent”. The queued emails are never sent: they stay in the queue with 0 send attempts, and since the record is “Sent”, the send job never picks them up again.

This affects practically every mailing that is planned in advance, as there is almost always a send run between queueing and the run date.

The failure is completely silent:

  • No error in any log file (prod.log, suitecrm.log)
  • Email Marketing Diagnostics shows everything OK
  • The status “Sent” suggests the mailing went out, while no recipient received it. The “Sent/Attempted” chart stays at 0.

Test emails are not affected, as they bypass the queue.

This is a regression: in SuiteCRM 7.x, mailings scheduled in advance were sent correctly.

Possible Fix

The record is set to “Sent” in
core/modules/Campaigns/Service/Email/Queue/DefaultEmailQueueProcessor.php:

if (empty($nextQueueEntries) && $isQueueingFinished) {
    $this->setSent($emRecord);
}

getEntriesToSend() in DefaultEmailQueueManager.php only returns entries whose send date has been reached ($now). Entries that are queued but not yet due are therefore treated as “nothing left to send”.

Possible fixes:

  • Only call setSent() if there are no remaining queue entries for the record at all (not just no due entries), or
  • Do not process the record in the send job before its scheduled run date, or
  • Do not queue emails before the scheduled run date.

Related: Campaigns get marked incorrectly as Sent · Issue #868 · SuiteCRM/SuiteCRM-Core · GitHub
(same code path, triggered there by failed sends / wrong SMTP credentials)

Steps to Reproduce the Issue

1. Make sure the schedulers "Queue Campaign Emails" and "Send Campaign Emails" run every 15 minutes (e.g. at :00, :15, :30, :45)
2. Create a campaign with a target list and an Email Marketing record
3. At 11:29, schedule the record with run date 11:50
4. At 11:30, the queue job queues the emails (before the run date)
5. At 11:45, the send job finds no due entries and sets the record to "Sent" (before the run date!)
6. After 11:50, the emails remain in the queue with 0 send attempts and are never sent

Counter-test with the same target list:
1. At 11:50, schedule a record with run date 11:55
2. At 12:00, the emails are queued and sent correctly (run date already passed when queueing)

Context

We migrated from SuiteCRM 7.x to 8.10.2. Since then, our planned newsletters were silently not sent, while the records showed “Sent”. The only reliable workaround is to schedule mailings with a run date of “now” or slightly in the past, so that emails are due when they are queued. Planning newsletters in advance is currently not possible.

Version

8.10.2

2

Issue, Version with better understanding the root cause

When the database server runs in a non-UTC time zone (e.g. MariaDB/MySQL with time_zone = SYSTEM and system time zone CET), scheduled Email Marketing records are set to “Sent” without sending any email, and the queued emails are never sent.

The send job mixes two different clocks:

  • DefaultEmailMarketingManager::getRecordsForQueueProcessing() selects records with
    ->andWhere('mkt.date_start <= NOW()') — the database’s local time (NOW()), while date_start is stored in UTC.
  • DefaultEmailQueueManager::getEntriesToSend() selects queue entries with
    send_date_time <= :now, where $now = $timedate->nowDb() — UTC from PHP.

With the DB at UTC+1, a record is picked up by the send job up to one hour before its run date (two hours during DST). At that point the emails are already queued (the queue job has no date filter and sets queueing_status = finished), but none of them is due yet. DefaultEmailQueueProcessor::processQueue() then hits:

if (empty($queueEntries) && $isQueueingFinished) {
    $this->setSent($emRecord);
    continue;
}

The record becomes “Sent” and is never selected again (status filter scheduled, pending_send, sending). The queued emails remain in emailman with 0 send attempts forever.

The failure is completely silent:

  • No error in any log file (prod.log, suitecrm.log)
  • Email Marketing Diagnostics shows everything OK
  • The status “Sent” suggests the mailing went out, while no recipient received it. The “Sent/Attempted” chart stays at 0.

Test emails are not affected, as they bypass the queue. This is a regression: in SuiteCRM 7.x, mailings scheduled in advance were sent correctly on the same server.

NOW() is the only place in core (apart from one migration) where the current time is taken from the database instead of PHP, which is why this goes unnoticed on UTC databases.

Possible Fix

  1. Use the same clock as everywhere else (root cause):
   ->andWhere('mkt.date_start <= :now')
   ->setParameter('now', $timedate->nowDb())
  1. Defensive: in processQueue(), only call setSent() if there are no queue entries left for the record at all (countEntries() === 0), not just no due entries. This also covers related cases such as #868.

Related: Campaigns get marked incorrectly as Sent · Issue #868 (same code path, triggered there by failed sends / wrong SMTP credentials)

Steps to Reproduce the Issue

Precondition: database not in UTC. Check with
SELECT NOW(), UTC_TIMESTAMP(), @@global.time_zone, @@system_time_zone;
In our case: NOW() = 15:26, UTC_TIMESTAMP() = 14:26, time_zone = SYSTEM, system_time_zone = CET.

  1. Make sure the schedulers “Queue Campaign Emails” and “Send Campaign Emails” run every 15 minutes (e.g. at :00, :15, :30, :45)
  2. Create a campaign with a target list and an Email Marketing record
  3. At 11:29 (local time), schedule the record with run date 11:50
  4. At 11:30, the queue job queues the emails
  5. At 11:45, the send job selects the record (NOW() of the DB is already past the run date in UTC terms), finds no due entries and sets it to “Sent”
  6. After 11:50, the emails remain in the queue with 0 send attempts and are never sent

Counter-test 1 (same target list): schedule at 11:50 with run date 11:55 → at 12:00 the emails are queued and sent correctly (no send run between queueing and run date).

Counter-test 2: with the database set to UTC (default-time-zone = '+00:00'), the scenario above sends correctly.

Context

We migrated from SuiteCRM 7.x to 8.10.2. Since then, our planned newsletters were silently not sent, while the records showed “Sent”. Workaround: run the database in UTC, or schedule mailings with a run date of “now” or slightly in the past.

Version

8.10.2 (code identical on current develop)