Importing emails automatically

I could not find such a topic from the recent time, so let me create one.

I think I checked everything that was able to find the documentation and my automatic mail import does not seem to be working.
The scheduler:

Cron tab:

allowed_cron_users in the config.php
image

Incoming mail setup is set to import email seems to correct as I was able to pull the folder names:

And I checked to import email automatically:

The syslog shows that the job is running every minute:

And the incoming mail keeps pending all the time and I need to import it manually:

There is an account with the email address from which the incoming mail comes:

But while importing manually I am getting this in the suitecrm.log:

What format is it about? Where? In the contact record?

Do you have the the email “type” set to group? SuitCRM only imports group emails, not personal.

OK, I think I finally got where the confusion came from.

Correct me if I am wrong: you are not talking here about SuiteCRM setup itself, but the mail system (external to SuiteCRM), right? So, I need to create a mailbox on the SMTP/IMAP server (I used Postfix/Dovecot on another Linux machine) that needs of a “group” type, right?

I thought you were referring to the IMAP mail import configuration in SuiteCRM that has to be of a “group” type.

Your original thought was correct. The imap type must be “group” or email will not be automatically imported.

Has nothing to do with email server settings the only thing that governs whether emails are imported automatically is that it is type group and its checked to import automatically. “Personal” type inbound email accounts do not get imported. Personal type emails are only a client and must be imported manually. Hope that makes it clear. See screen shots below.

Hi again!

I recorded a short video to show you my setup.

I really do not understand what I am doing wrong… The Cron job just does not fire these imports at all :frowning:

Your cron jobs are failing. Even though they show as running, your job to fetch inbound emails is not running your logs show that.

You need to troubleshoot why your cron jobs are not succeeding.

Some common issues are:

  1. Not enough memory allocated. (#1 reason!)
  2. One or more of your inbound email boxes are not configured correctly causing the job to fail and the imap errors you are getting. I’d suggest making on one inbound email account active so you can narrow down which one.
  3. Cron job is not allowed to run by server user. You can check your config to see which cron users are allowed.
  4. File permissions, this one is pretty easy to rule out by resetting your file permissions.

(as part of troubleshooting, are other scheduled jobs running like workflow for example?).

Just to clarify again, ONLY group accounts will import emails. Personal accounts are only a client. They fetch a list of emails, but do not import them to the databases (unless you do it manually one by one).

I wouldn’t wory about the relationship warnings, I get those all the time I haven’t really found any negative concequences.

One trick I do is comment out the CLI only stuff in cron.php. Then that will allow you to run the cron from the URL https://yoururl.com/cron.php

put this at the top of cron.php

error_reporting(E_ALL);
ini_set('display_errors', 1);

Then when you navigate to the url, it will run the job and output the errors why your cron if failing. This usually gives a pretty good idea what to fix.

The other thing to do is check all your scheduled jobs and see if you see one that fails. This often prevent the rest from running. You can disable that one and the rest should run. Then at least you’ve narrowed it down to one job and you can trouble shoot that.

(don’t forget to put CLI stuff back, or someone and DDoS your server by hitting the link.)

Hi!
Yes, I have spent a couple of hours on troubleshooting, and long story short-- the Check Inbound Mailboxes job record was damaged somehow and the cron.php was just ignoring it (did not process it at all), so I opened the table in DBeaver and made it look like the ones that were working correctly. Then I opened it from the web UI and re-saved it, and bum! It started to work!!! Great step forward!

So, now the emails are being automatically imported as expected! Which is great!

But of course, SuiteCRM would not be SuiteCRM if I did not find another bug in it :slight_smile:

The emails that are auto-imported are not fully functional emails. When I open them from a Contact form (from the Insights panel on the right) … they go blank. There is no field visible whatsoever:

Contact form:

Imported email:

So, again – I opened my database client (DBeaver) and compared two emails :

  • an email that was imported manually, which opening just fine (all info shows up)
  • an email that was automatically imported-- which is not OK (blank)

And here was the difference:


parent_type and parent_id fields were empty!

After copying the parent_type field value into the bad email record-- another bum! The email started to open just great:

So, the function most likely returnImportedEmail (supposedly) which is a part of the InboundEmail.php file is screwed and it does not set these two fields as the function for the manual manual import does.

So, my question is now: can you or somebody else let the developers know about this nasty bug, please?

Or should I start developing my own SuiteCRM fork? :slight_smile:

P.S. Yes, I know that ONLY group accounts will import emails and that the personal accounts are only a client :slight_smile: I am sorry I have not made it clear enough.

1 Like

I think I saw a couple of posts about this bug. See issues on github and see if the bug is logged and/ or fix proposed.

1 Like

What is the applicable logic to determine which parent should be filled for that email, when being imported automatically?

I am wondering that perhaps the code isn’t filling that in because it isn’t obvious (no way to find out, or there is more than option to relate the record, and no way to know which one you would want).

These things can get complicated when you consider that an account can have several email addresses, and several contacts, each one with several addresses; and these addresses can be repeated across different entities… etc.

This would be a good job for AI to figure out what’s the most suitable record to relate this email message to, and let it set these values parent_type and parent_id.

I haven’t delved in to the “how” too much but based on experience it looks for a matching email and relates it to the contact. I can’t think of an example where it would get related to the account and not the person. I’ll try that out when I get a chance.

Ok did some testing. Automatic import of group emails does not work. It shows that they have been imported with the checkmark, but they are not related to anything. This seems to be broken.

Case emails seem to work fine, import and attach to the correct contact and/or case.

Yes, you are right, but in my case there is only one entry having this email address, so there is no confusion.

hey @mwojtaszek it does not work in any case. Whether contact email or account email it never relates it to anything. It shows as imported but not related to anything. This is definately broken. I didn’t see anything specific as a bug on Github so this must be something new.

Ok this looks like another can of worms! It looks to me like the function: handleLinking() in InboundEmail.php is the one that supposed to link the emails to contacts. However, I don’t see it being called anywhere when I search for it on Github? I’m thinking it needs to get called in the loop in pollmonitorediboxes() in _AddJobsHere.php

I wrote some code that kind of fixes it, but I am swamped with something else right now and I do have much time to test it thoroughly.

I was kind of thinking as a quick fix, you could probably do an on save hook or workflow that just relates the emails to the contact.

Here’s my algorithm that should make Importing Group Emails Automatically, work again:

  1. get the From email address, and From name, from the email message.
  2. Is there a Contact record that is associated with this From email address?
    • Yes: set parent_type to Contacts, and parent_id to the ID of the Contact that is associated with this email address. We’re done.
    • No: Continue to the next step.
  3. Is there any Module record that is associated with this From email address?
    • Yes: set parent_type to the type of that Module record, and parent_id to the ID of that Module record that is associated with this email address.
    • No: Create a new Lead, set the Person’s name to the From name from this email message, associate a new Email address, associate it to this new Lead, and set parent_type to Leads and parent_id to the ID of this new Lead.
1 Like