Emojis keep getting changed to question marks when creating an email template 8.9.1

Sorry admins I had to repost with full details:

CentOS Linux 7 (Core) | Apache/2.4.6 | PHP 8.3.8 | SuiteCRM version 8.9.1 | MariaDB version 11.1.6 | PHP 8.3.8

Its a near-fresh installation just made a couple changes within the suitecrm ui

The problem:

First post on the forums! Thanks to anyone reading - would love to get to the bottom of this problem.

I am trying to create an email template and my emojis keep getting changed to question marks “???”

While editing:

After saving:

Here is what I have tried so far:

The database was originally in utf8mb3, I converted all tables to be utf8mb4 and utf8mb4_unicode_ci

Then I changed public/legacy/config.php and changed the following two lines under dbconfig from NULL

    'collation' => 'utf8mb4_unicode_ci',
    'charset' => 'utf8mb4',

Then did Admin→Repair→Quick Repair and Rebuild

I also cleared public/legacy/cache/*

For some reason though emojis still keep getting changed into question marks

I verified the db is not the issue by creating a table and inserting an emoji - all looks good

MariaDB [db]> select * from emoji_test;
+----------------+
| txt            |
+----------------+
| 💫 🫖 👉 |
+----------------+

I even tried changing an email template directly through the db but the output on suitecrm was still not an emoji, albeit it wasn’t question marks this time:

MariaDB [db]> UPDATE email_templates 
    -> SET body_html = '<p>This is a test 💫</p>' 
    -> WHERE name = 'ATester';

Not sure what the issue is or what to try next…

If you need any further info please let me know

Yes, maybe it is not allowed in the SuiteCRM.

Check out the locale menu under admin.

Also, check the config.php file, maybe it has some parameter related to charset.

Hello,

I’ve just tested it on 8.9.2 and it’s still the same issue:

However, on 7.15 it’s already fixed:

There has been a TinyMCE upgrade recently on the 7.x branch:

TinyMCE 8: The text editor has been upgraded to provide a consistent look and feel across all SuiteCRM modules.

And I hope that all the neat new features - including the new TinyMCE editor - will be part of the 8.10 release:

Thanks for the reply,

I understand what you are saying - however I tried to put emojis on a non tinymce field (see below) and had the same result

image

The “???” was an emoji

So now I’m wondering does this even have anything to do with tinymce at all? :thinking:

Around similar topic:

I managed to fix the problem - it wasn’t a tinyMCE issue

I did two things and I don’t know what fixed it.

  1. I changed the mariadb system variable “Old Mode” to be blank

  2. I then changed from utf8mb4_unicode_ci to utf8mb4_general_ci, SuiteCRM8 comes shipped with utf8mb3_general_ci so it makes sense to go from “general” to “general”.

I did “2” using the same steps as before.

I first changed the public/legacy/config.php and set the following under “dbconfig” section

‘collation’ => ‘utf8mb4_general_ci’,
‘charset’ => ‘utf8mb4’,

I then did a query and changed every single table to use utf8mb4 charset and utf8mb4_general_ci collation, I also set this as the default charset and collation

I then did Admin→Repair→Quick Repair and Rebuild

Then went on the terminal and did the following

rm -rf public/legacy/cache/*
php bin/console cache:clear

Now it’s all working fine :+1: (Existing ??? did not get turned back into emojis though)

1 Like

You can raise an issue on GitHub.