Email import failure on new install

[kenneth]

Hi,
I’m exploring SuiteCRM for the first time and it looked like I had a successful install, but maybe not. I tried to set up a group incoming email account, and it looks like it’s connected successfully. I see the emails in Emails → View Email. But import does not work, either automatic or manual.

I read some other posts suggesting that an install might fail partially, leaving some features broken, so I reran the install script (after deleting config.php) and still have the same behavior. After some digging, I found the following in my php log:

[28-Oct-2024 22:56:53 America/Detroit] PHP Warning:  Attempt to read property "subtype" on null in /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php on line 5550
[28-Oct-2024 22:56:53 America/Detroit] PHP Fatal error:  Uncaught TypeError: SuiteCRM\HtmlSanitizer::clean(): Argument #1 ($dirtyHtml) must be of type string, null given, called in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php on line 116 and defined in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php:138
Stack trace:
#0 /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php(116): SuiteCRM\HtmlSanitizer->clean(NULL, true)
#1 /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php(4154): SuiteCRM\HtmlSanitizer::cleanHtml(NULL, true)
#2 /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php(5548): InboundEmail->getMessageTextWithUid('11', NULL, NULL, NULL, true)
#3 /var/www/suitecrm/public/legacy/modules/Emails/EmailsController.php(599): InboundEmail->returnImportedEmail('11', '11')
#4 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(522): EmailsController->action_ImportAndShowDetailView()
#5 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(491): SugarController->do_action()
#6 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(465): SugarController->handle_action()
#7 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(361): SugarController->process()
#8 /var/www/suitecrm/public/legacy/include/MVC/SugarApplication.php(101): SugarController->execute()
#9 /var/www/suitecrm/public/legacy/index.php(52): SugarApplication->execute()
#10 {main}
  thrown in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php on line 138

Thinking there may be an issue with IMAP, I tried both with SSL and without, but that didn’t make a difference. I am using SuiteCRM 8.7.0, PHP 8.2.24. Any suggestions would be wonderful.

[nhat.thieu]

Can you tried with PHP 8.1.25, Looks like it works for me. Basically, I think PHP newer version was force type of value exactly then able to use the function. So you can try again with older for example 8.1 may it work both null and string value even the function input required as string

[kenneth]

Thanks for the suggestion. I got the same error when trying to auto-import from the group inbox with PHP 8.1.30.

However, as I continued to test things, I was able to manually import an email from an individual inbox. Trying to figure out what the difference was, I realized that auto-import from the group inbox will actually work (or at least do something) if the email includes special formatting codes (HTML or RTF, it seems). My plaintext test messages continue to fail.

Confusing. But with this new insight, I switched back to PHP 8.2, and I see the same behavior with either version.

[kenneth]

I corrected the error by adding a line in public/legacy/include/HtmlSanitizer.php though I have to imagine that there’s something else going on that causes this. Anyway, it’s working for the moment after adding the if statement below, around line 116:

    public static function cleanHtml($dirtyHtml, $removeHtml = false)
    {
        if (!is_string($dirtyHtml)) $dirtyHtml = '';
        return self::getInstance()->clean($dirtyHtml, $removeHtml);
    }

[pgr]

Can you get us a stack trace at that point in the code, so I can check how it is being called?

[kenneth]

Hi, I included a stack trace in the first post under this topic. Is there something more I can provide to give better detail?

[pgr]

@kenneth my apologies, I missed that, my bad

Do you have any other relevant error immediately before that one, even if it isn’t a FATAL? It seems as though the $emailMessage being “cleaned” is null, which breaks the code in cleanHTML because it isn’t expected; but the real problem is probably a bit before when it fails to get a valid emailMessage in the first place (and fails to show a proper error to the user…)

[kenneth]

This is what shows up in my suitecrm.log file (without the fix), on Log Level “Warn”:

Sat Nov  2 15:30:02 2024 [2598100][1][WARN] Request ssl value not found.
Sat Nov  2 15:30:02 2024 [2598100][1][WARN] Requested folder is not defined
Sat Nov  2 15:30:02 2024 [2598100][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 15:30:02 2024 [2598100][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 15:30:02 2024 [2598100][1][FATAL] Job f0dd2ba7-25a6-29df-96ad-67267dda5a88 (Check Inbound Mailboxes) failed in CRON run

If I try to manually import the email (note that even for emails that fail import, the body of the emails still show up under Emails → View Email prior to import), I get the following in my PHP error log:

[02-Nov-2024 16:10:38 America/Detroit] PHP Warning:  Attempt to read property "subtype" on null in /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php on line 5550
[02-Nov-2024 16:10:38 America/Detroit] PHP Fatal error:  Uncaught TypeError: SuiteCRM\HtmlSanitizer::clean(): Argument #1 ($dirtyHtml) must be of type string, null given, called in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php on line 116 and defined in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php:138
Stack trace:
#0 /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php(116): SuiteCRM\HtmlSanitizer->clean(NULL, true)
#1 /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php(4154): SuiteCRM\HtmlSanitizer::cleanHtml(NULL, true)
#2 /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php(5548): InboundEmail->getMessageTextWithUid('21', NULL, NULL, NULL, true)
#3 /var/www/suitecrm/public/legacy/modules/Emails/EmailsController.php(599): InboundEmail->returnImportedEmail('8', '21')
#4 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(522): EmailsController->action_ImportAndShowDetailView()
#5 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(491): SugarController->do_action()
#6 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(465): SugarController->handle_action()
#7 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(361): SugarController->process()
#8 /var/www/suitecrm/public/legacy/include/MVC/SugarApplication.php(101): SugarController->execute()
#9 /var/www/suitecrm/public/legacy/index.php(52): SugarApplication->execute()
#10 {main}
  thrown in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php on line 138
[02-Nov-2024 16:10:39 America/Detroit] PHP Warning:  Attempt to read property "subtype" on null in /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php on line 5550
[02-Nov-2024 16:10:39 America/Detroit] PHP Fatal error:  Uncaught TypeError: SuiteCRM\HtmlSanitizer::clean(): Argument #1 ($dirtyHtml) must be of type string, null given, called in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php on line 116 and defined in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php:138
Stack trace:
#0 /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php(116): SuiteCRM\HtmlSanitizer->clean(NULL, true)
#1 /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php(4154): SuiteCRM\HtmlSanitizer::cleanHtml(NULL, true)
#2 /var/www/suitecrm/public/legacy/modules/InboundEmail/InboundEmail.php(5548): InboundEmail->getMessageTextWithUid('21', NULL, NULL, NULL, true)
#3 /var/www/suitecrm/public/legacy/modules/Emails/EmailsController.php(599): InboundEmail->returnImportedEmail('8', '21')
#4 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(522): EmailsController->action_ImportAndShowDetailView()
#5 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(491): SugarController->do_action()
#6 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(465): SugarController->handle_action()
#7 /var/www/suitecrm/public/legacy/include/MVC/Controller/SugarController.php(361): SugarController->process()
#8 /var/www/suitecrm/public/legacy/include/MVC/SugarApplication.php(101): SugarController->execute()
#9 /var/www/suitecrm/public/legacy/index.php(52): SugarApplication->execute()
#10 {main}
  thrown in /var/www/suitecrm/public/legacy/include/HtmlSanitizer.php on line 138

and here’s the contents of suitecrm.log around the same time including entries before and after:

Sat Nov  2 16:10:24 2024 [2594060][1][DEPRECATED] Array
(
)

Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'Feeds' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'iFrames' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'TimePeriods' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'ContractTypes' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'Notifications' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'Queues' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'TrackerSessions' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'TrackerQueries' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'FAQ' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'Newsletters' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'SugarFavorites' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'Library' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'KBDocuments' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'jjwp_Partners' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'TemplateEditor' not mapped to 'frontend'
Sat Nov  2 16:10:24 2024 [2594060][1][WARN] Cannot find bean file for module: CustomFields
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] Cannot find bean file for module: CustomFields
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] Cannot find bean file for module: Versions
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] Cannot find bean file for module: Versions
Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates_idb for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates_idb for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship projects_contacts_resources
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] project_resource for projects_contacts_resources failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship am_projecttemplates_contacts_resources
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] am_projecttemplates_resources for am_projecttemplates_contacts_resources failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates_idb for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2594060][1][ERROR] Unable to find relationship emails_email_templates
Sat Nov  2 16:10:25 2024 [2594060][1][WARN] emails_email_templates_idb for emails_email_templates failed to load

Sat Nov  2 16:10:25 2024 [2598151][1][WARN] ModuleNameMapper | mapName | 'emails' not mapped to 'frontend'
Sat Nov  2 16:10:25 2024 [2595734][1][DEPRECATED] Array
(
)

Sat Nov  2 16:10:25 2024 [2595734][1][WARN] CSS File Dawn/yui.css not found
Sat Nov  2 16:10:25 2024 [2595734][1][WARN] Request ssl value not found.
Sat Nov  2 16:10:25 2024 [2595734][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 16:10:25 2024 [2595734][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 16:10:25 2024 [2595734][1][WARN] Email ID is Empty
Sat Nov  2 16:10:25 2024 [2598151][1][DEPRECATED] Array
(
)

Sat Nov  2 16:10:29 2024 [2598152][1][WARN] Email ID is Empty
Sat Nov  2 16:10:38 2024 [2598151][1][WARN] Request ssl value not found.
Sat Nov  2 16:10:38 2024 [2598151][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 16:10:38 2024 [2598151][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 16:10:39 2024 [2594060][1][WARN] ModuleNameMapper | mapName | 'emails' not mapped to 'frontend'
Sat Nov  2 16:10:39 2024 [2595734][1][WARN] Request ssl value not found.
Sat Nov  2 16:10:39 2024 [2595734][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 16:10:39 2024 [2595734][1][FATAL] ImapHandler trying to use a non valid resource stream.
Sat Nov  2 16:11:00 2024 [2598151][1][DEPRECATED] Array
(
)

I hope this is helpful.

[pgr]

I can’t figure it out, these email things are tricky in SuiteCRM unless you go full debug mode with an IDE and Xdebug, stepping through the code…

This issue seems different from yours, but you have some log messages in common. Have a look to see if any of that discussion seems helpful to you:

Subject Line Missing from Inbound Emails

Please help. I have recently installed Suitecrm and have followed a guide to integrate Suitecrm with Microsoft 365. I am trying to sync emails from office 365 mail with Suitecrm via OAuth. I have set up the External OAuth Provider, a connection, and an inbound email account. As shown in the image, my inbox in Suitecrm populates with the emails, but the subject line is missing and the emails cannot be opened. I am using Ubuntu 22.04, Suitecrm 8.2.3, PHP 8.0.28, Apache 2.4.52, and MariaDB 10.6.1…

You seem to be hitting some difficulty or bug opening up the IMAP folder, and from then on it’s just a sequence of FATALs due to that original problem. But I don’t know why that happens in the first place

@pstevens if you have any ideas, please chime in

[pstevens]

I did a lot of work with Group Emails, but mostly to get users other than admin to be able to see them. This is a different problem. I’ll try and see if I can replicate the issue in one of my dev environments.

[pstevens]

Just tested on 8.6.2 on PHP 8.2. Group email accounts import emails without issue. ( I don’t have an O365 account to test with, it’s just a regular cpanel imap email account)

[kenneth]

I’ve been studying the code trying to figure out what’s going on, and I think there’s something fishy with setting $email->description_html on line 5550 of modules/InboundEmail/InboundEmail.php. I can’t say I fully understand what the intent is here, but I wonder if this should be in an else block attached to the previous if. In my (failing) case, the Content-Type is ‘text/plain’ so there’s no HTML to work with. Further, it’s not a multipart message, and (?) $structure->subtype is null (I think?).

My theory is that the call to getHtmlContent() on line 4150 is returning null because there is no HTML content to get (since the message is text/plain—but this code doesn’t see it as text/plain because the type sent to the function is null). That, subsequently, is causing the error on line 4154.

I don’t know why this error is only showing up for me, not for @pstevens . But note that I am on 8.7.0, not 8.6.2. It’s also possible that my IMAP server is working differently than others? but I’m not running any fancy software, just dovecot (which I believe is the same as cpanel).

[pstevens]

Ok I can confirm. Plain text emails do not import.

[pgr]

I suggest changing this

to this (adding a check to see if description_html is there):

… although that might still not work if it creates something unexpected, somewhere else. But it’s worth a try, and after some more tweaking it should get your through.

I am afraid that text-only emails are quite untested in many parts of SuiteCRM. I’ve been guilty of that negligence myself