There was a suggestion box / idea graveyard on Trello, lasted for a couple of years, then disappeared, along with the hundreds of suggestions buried therein.
The devs know about most of this, but they are outnumbered, like, a thousand to one. So much to do, nobody to do it.
Email got important improvements recently, the email accounts management is less flaky. For the rest I would bet on incremental improvements. I would be doing quite a lot here if PRs were getting merged, but theyāre not.
(my PowerReplacer add on can be used as a sort of low-code language to make inbound email process email intelligently, linking them to records, assigning them, notifying people, testing conditions, etc. You can do amazing stuff when you start getting control of email, and you have a full-blown corporate CRM behind you)
@pstevens ā work on investigating these complex areas of SuiteCRM has been amazing.
LOL Iām just looking at this part now! So basically, ANY email that is returned from MAILERDAEMON or POSTMASTER is marked as ābouncedā. There is a secondary procedure which Iām working through now which determines āWhyā it bounced.
Basically, the checBouncedEmailInvalid function returns ātrueā IF ā¦ lines in the email description that start with āStatus:,ā followed by a sequence of three numbers separated by periods. If it finds a line with a permanent error status (5.X.X) and a line that contains āDiagnostic-Code: smtp; 550,ā it returns true to indicate that the email is invalid due to a ā550ā SMTP error.
Once it returns true then another function: createBouncedCampaignLogEntry marks $bounce->activity_type = āinvalid emailā otherwise, it marks it as āsend errorā. So far pretty much everything Iāve tested gets marked as āsend errorā. Based on the fact that itās only looking at the two senders (or an additional one Iāve specified), maybe this could be relaxed a bit and look for the words āerrorā and ā550ā, you may get some false positives, but might be way better than getting almost no matches.
After that write-up, what would be your suggestions for future improvement of this in SuiteCRM? Weāve already discussed the ability to configure the regular expression that matches bounces, what else do you think would be helpful?
One other thing that would be great, is if there was an API for each of the major outbound mailers like: sendgrid, AWS, Elastic Email, etc. The API bounce handling is much more accurate if you get it right from the sender via API. Mautic gives the user a choice of doing a bounce account like SuiteCRM or selecting one of the popular bulk mailers from a dropdown and just entering the API info.
This is so much easier, then you donāt have worry about the bounce account at all if your primary outbound mailer is a third party like Sendgrid or AWS. They all pretty much have API integration that returns the bounce info.
I had a look at trying to add the different sender as a variable one could enter in the Inbound mail account, then contributing the code to Github. Adding the variable to the function is not a problem. Adding the field to front end editview/detailview I think I could do, but adding a new field to SuiteCRM base code to be included when installed and updated would be a giant stretch for me. Not really sure even where to start with that.
To send I have found (specially with v8.4, much better than the previous versions) that AWS for example works great.
Configure the SMTP server, username and password which are the credentials given by AWS SES and good to go.
Complaint and Bounce handling by Amazon SES is done using Amazon SNS (simple notification system) it requires some configuration. And it doesnāt work with returned emails, bodies and headers from what I remember on top of my head.
From what I understood also, (and please do correct me if Iām wrong) the bounce email handling account, that does have to be added to SuiteCRM and then selected when building a campaign requires it to be an actual inbound mailbox, but that mailbox is on your server not an AWS inbox.
So I guess it could be interesting yes, but considerations Iād say how the email account we select when building campaigns will be considered when using such plugins, if a message bounces and is delivered to said inbox, when a notification has already been generated, will it create a double bounce or something?
Subscription protocols available for SNS are SQS, Lambda, HTTP, SMS, email and mobile endpoints.
@maverickws Iāve never tried AWS specifically, but my understanding from their documentation is that if you set the bounce path and check off āinclude original headersā it will return the original header containing the campaign id.
So what happens when you select a bounce handling account in the campaign is that it includes a bounce path in the header or the emails which contains your bounce email address which typically is different from the reply to. Most servers honor this and send bounces to the bounce return path specified. However, bulk emailers like AWS, Sendgrid, etc. have their own bounce handling systems and do not honor your bounce path and replace it with their own. Soā¦ itās necessary to configure them to send the bounce notifications to your bounce account. (configuring just in SuiteCRM wonāt work because it gets stripped out).
Sendgrid includes the original header by default with the campaign id, but AWS you need to specify that, there is an option for it. (Again, Iām going by reading the documentation but Iād love someone to chime in if that is in fact the case).
Ok. I might have to revisit SES and SNS donāt remember some stuff. So trying to understand the final result we get
Configure a bounce SNS topic to deliver (format, email) to the bounce mailbox.
On SuiteCRM
Set Amazon SES SMTP on outbound,
the bounce mail account on incoming
create the campaign, set the bounce mailbox (that will get the bounces handled by amazon sns).
And figuring the Amazon bounce configurations supersede bounce configuration that may be sent by Suite?
Yes, when you send email through a 3rd party like Amazon, Sengrid, Mailgun, etc. they supercede the original mail headers and insert their own bounce path. So you have to configure the 3rd party sender to return bounces to your bounce account (with original headers).
SuiteCRM will process the bounce account via cron job and create the bounce entries in the campaign.
The other problem with third party senders is they often donāt return the email from POSTMASTER or MAILER-DAEMON, so SuiteCRM doesnāt parse them. (Sendgrid is an example, which is why you have to mod the line of code in check for bounced emails). However, I can confirm that AWS sends back from MAILER-DAEMON @amazones.com so SuiteCRM will parse it as long as it is delivered to the bounce account with the campaign ID in the header (by selecting the include header option).
This is all theory, since I have not tried AWS specifically, but if you do, please share results. Also if can share screen shots or steps to add the right bounce account to AWS, Iād love to include those in my article too!
OK so I have an update with AWS. When you configure SNS in AWS to send bounce notifications you have the option to include original header, which is great because you get the camapaign ID back.
However, there are two problems:
The email comes from no-reply@sns.amazonaws.com, not a big issue I can edit SutieCRM to look for this sender.
Unfortunately, it comes in JSON format and not plain text. Iāll have to test and see if SuiteCRM still finds what it needs or I have to JSON decode before SuiteCRM can parse it.
Bookmarked! Thanks!
ahah. Iāll give this a go not sure today.