Email can not view bounce detail

[gbcbooks]

I have installed Suitecrm 8.5 successful in a docker.

now everything runs well execpt two

1. one is as follow
   
   
   
   this comes with error log like this:

[2024-02-11 14:10:05] request.ERROR: Uncaught PHP Exception Symfony\Component\HttpKernel\Exception\AccessDeniedHttpException: "Invalid CSRF token" at /var/www/html/core/backend/Security/CSRFValidationListener.php line 95 {"exception":"[object] (Symfony\\Component\\HttpKernel\\Exception\\AccessDeniedHttpException(code: 0): Invalid CSRF token at /var/www/html/core/backend/Security/CSRFValidationListener.php:95)"} []

2. two is some bounce email create by my own mailserver can not view detail on suitecrm EMAIL Module
   i offer sample picture here:
   
   
   
   
   
   
   this error comes with log:

2024/02/11 06:12:44 [error] 28#28: *138 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught TypeError: Argument 1 passed to SuiteCRM\HtmlSanitizer::clean() must be of the type string, null given, called in /var/www/html/public/legacy/include/HtmlSanitizer.php on line 116 and defined in /var/www/html/public/legacy/include/HtmlSanitizer.php:138
Stack trace:
#0 /var/www/html/public/legacy/include/HtmlSanitizer.php(116): SuiteCRM\HtmlSanitizer->clean()
#1 /var/www/html/public/legacy/modules/InboundEmail/InboundEmail.php(4154): SuiteCRM\HtmlSanitizer::cleanHtml()
#2 /var/www/html/public/legacy/modules/InboundEmail/InboundEmail.php(5807): InboundEmail->getMessageTextWithUid()
#3 /var/www/html/public/legacy/modules/Emails/include/DetailView/EmailsNonImportedDetailView.php(89): InboundEmail->returnNonImportedEmail()
#4 /var/www/html/public/legacy/modules/Emails/views/view.detailnonimported.php(77): EmailsNonImportedDetailView->populateBean()
#5 /var/www/html/public/legacy/include/MVC/View/SugarView.php(208): EmailsViewDetailNonImported->preDisplay()
#6 /var/www/ht" while reading response header from upstream, client: 172.27.0.1, server: localhost, request: "GET /legacy/index.php?folder_name=INBOX&folder=inbound&inbound_email_record=cacd62ca-8c3c-6ad0-bb73-65c7914b24b6&uid=18&msgno=18&module=Emails&action=DisplayDetailView HTTP/1.1", upstream: "fastcgi://172.27.0.3:9000", host: "crm.abc.com", referrer: "https://crm.abc.com/"

[gbcbooks]

For the email can not be view problem, my research is:

HtmlSanitizer.php->clean() function can’t clear dirtyHtml, the regexp pattern may be wrong in some place, so the dirtyHtml became empty when return.

and that is why SuiteCRM can’t view bounced messages.

here is my solution:
thanks for the article from: Problem parsing Inboud emails when email is multipart/mixed, but has only text/plain and attachment parts, but not text/html · Issue #8505 · salesagility/SuiteCRM · GitHub

edit InboundEmail.php->getMessageTextWithUid()

        # fix problem of suitecrm can't view bounced message from postfix system
        # orignal code
        #$emailMessage = $this->mailParser->parse($emailBody, false)->getHtmlContent();
        # replace with start
        $emailTextMessage = $this->mailParser->parse($emailBody, false)->getTextContent();
       	$emailHtmlMessage = $this->mailParser->parse($emailBody, false)->getHtmlContent();
        if(!empty($emailHtmlMessage)){
			$emailMessage = $emailHtmlMessage;
		}else{
			#$emailMessage = $emailTextMessage;
            $emailMessage = $emailBody;
		}
        # replace with end

[pgr]

Nice find. This should be proposed as a fix on Github, is it already there?

Maybe it’s one of the many pending @pstevens fixes?

[rsp]

Yeah, we need to upvote those GitHub PRs ASAP

[pstevens]

@pgr Not one of mine, I hadn’t run into this particular problem yet. I’ve got trying to fix project templates in my queue for my next big project as soon as I get a minute. Been busy the last couple of months. Project templates are not getting the start dates right when they are supposed to start concurrently with their predecessor if you create the project from a template. I think I know where the problem is, just need to figure out how to get that logic into it.

[rsp]

@pstevens

Project templates are not getting the start dates right when they are supposed to start concurrently with their predecessor if you create the project from a template.

Is it allowed to have two tasks with same start dates and times?

[pstevens]

Hey @rsp, if you want a thread related to Projects, might be a better idea to open a new one rather than get this one off topic. Sorry didn’t mean to take the thread in a different direction.

[gbcbooks]

this solution is not 100% perfect, because dirtyHtml is not clean.

But I think this can at least help others to get the project running first.

[ggraynoth]

Thank you so much.

I had been working with our hosting partner to work out why the bounceback function was not working due to the bounceback e-mails not being populated in SuiteCRM. We use email accounts on the hosted platform and assumed it may be some security function stopping SuiteCRM gaining full IMAP access to this type of email for some reason.

Not suprisingly, they couldn’t find anything wrong and I have been looking through the code for quite a while to try and track where this was going wrong, without success. I gave gave up in the end, as nothing came back from my forum post a while back.

Now I know why. I have looked at this very code and didn’t spot the problem, as all other emails were working fine, so I assumed this was parsing correctly, without understanding the finer detail around the email structure itself. So, as mentioned, a great spot from you and linking it to that almost 4-year old Github post.

I am going to apply the fix and run a dummy campaign on our test system. If this does the trick, it will solve one of our few outstanding issues, even if not upgrade safe. We have been handling boucebacks externally and have become borderline ‘expert’ with email export, Excel manipulation and upload to tag the contacts correctly.

I will let you know. Fingers crossed.

[ggraynoth]

Progress but still bounceback fails.

So, I did the update and used the $emailTextMessage line after the ‘else’, as the bounce emails typically do not have the email body repeated (usually returned as an attachement), just a text based message informing sender of the fail in various different forms. (Mailer-Daemon, Postmaster replies etc…) and hey presto, the bounced emails appear with the text content, as they appear in the externally connected IMAP mailbox. No more blank screens - Hoorah.

In the external mailbox, they also have one or two attachments with the original body and the error detail. These only come into the SuiteCRM mailbox for bounced returns if you do a manual ‘Import’ by selecting the emial and doing the import under actions.

So the entry in the bounced mailbox is the from,subject etc… and the body is the text based rejection text but no attachments.

Running several test campaigns, the Bounce handling still does not work, however, if you go into the bounceback email, it treats this as a normal email and tags it as read in the campaign status display if you open one of the bounces, so it knows it is there but is not recognising this as a bounce in the code.

So rather than bore you silly with all of the code digging that i have done in the ProcesseBouncedemail.php and associated code and functions, I am further forward but no nearer to understanding where it is failing.

What I did do was run the Scheduler process bounce function every 30 minutes, just to track it and as per the previous situation, I see the same group of fatal errors in the log every 30 minutes.

Mon Feb 19 10:30:02 2024 [6390][1][FATAL] ImapHandler trying to use a non valid resource stream.
Mon Feb 19 10:30:02 2024 [6390][1][FATAL] ImapHandler trying to use a non valid resource stream.
Mon Feb 19 10:30:02 2024 [6390][1][FATAL] An Imap error detected: “IMAP search error”
Mon Feb 19 10:30:02 2024 [6390][1][FATAL] ImapHandler trying to use a non valid resource stream.
Mon Feb 19 10:30:02 2024 [6390][1][FATAL] ImapHandler trying to use a non valid resource stream.
Mon Feb 19 10:30:03 2024 [6390][1][FATAL] An Imap error detected: “IMAP search error”

I know where they come from in ImapHandler.php, I can see the code that generates them but that is where my level of understanding runs out. This is '000s of lines of code

Many thanks for your help as this has really tidied up the bounced emails. So I will leave you in peace and persevere with my hunt for a fix, which may never be found .

Thanks again.

[ggraynoth]

OMG - it is working this morning. For the first time ever, the Bounce processing has worked. A 100 lead email campaign has caught every bounce and correctly tagged them in SuiteCRM lead records.

I have no idea what has happened this morning unless it was simply a cacheing issue, as I flushed the cache this morning and nothing else has changed since yesterday’s update as provided above.

Now all working - so a massive Thank You. I will treat the above as the solution. @pstevens - I will hopefully not need to bother you about bounces again. I am not going to touch anything

Thanks again.

[chris001]

@gbcbooks
Please click Solution on the reply that solved your issue, for other users to easily locate it.

[ggraynoth]

This was the one that solved my issue but no option to click, as no solution tag for some reason.

Email can not view bounce detail

For the email can not be view problem, my research is: HtmlSanitizer.php->clean() function can’t clear dirtyHtml, the regexp pattern may be wrong in some place, so the dirtyHtml became empty when return. and that is why SuiteCRM can’t view bounced messages. here is my solution: thanks for the article from: Problem parsing Inboud emails when email is multipart/mixed, but has only text/plain and attachment parts, but not text/html · Issue #8505 · salesagility/SuiteCRM · GitHub edit InboundE…