Somebody happy with SuiteCRM 7.12.2 and Elasticsearch?

isaacmarco · 24 January 2022 11:18

I’m working in order to migrate an old SuiteCRM 7.6.7 to the most recent release and it’s been hard.

Now I have a successful migration path with this steps…

SuiteCRM-Upgrade-7.6.x-to-7.8.31.zip
SuiteCRM-Upgrade-7.8.x-to-7.10.22.zip
SuiteCRM-Upgrade-7.10.x-to-7.12.2.zip

SuiteCRM 7.12.2 is supposed to be an LTS version and at the moment there is no upgrade package for SuiteCRM 8 so 7.12.2 was my target. By other hand 7.11 uses ElasticSearch 5 which it’s old and without support and 7.12.2 is supposed to be prepared for Elasticsearch 7 but with some problems than I was able to solve here https://github.com/salesagility/SuiteCRM/issues/9421

Every thing seems to be working OK after my migration to 7.12.2 but when I config Elasticsearch and the full index is finished I have many problems and and buggy results from the Global Search when looking for leads with spanish especial characters like “ñ” or accented vowels “á é í ó ú” and thats not usable.

I’m not sure but I think in 7.11 that was working for me OK, but I need to repeat my migration in order to confirm that.

Please… some help will be appreciated…

Have you migrated to 7.12.2? Are you waiting in 7.11? Somebody is working a new 7.12.(3) version solving these problems? There will be in the short time an upgrade package to upgrade to SuiteCRM 8?

I should decide this week my final target version and your opinions will be very appreciated because maybe is better to upgrade to the last 7.11 version and wait there for some time.

Thank you in advance.

CRMSigns · 24 January 2022 14:48

7.12 search out of the box is not great. 7.11 had Lucene global search built in and while not fast at least it worked.

I updated to 7.12 a couple weeks ago and was not happy to find the PDF engine changed (all pdf’s broke) and I could no longer effectively run a global search. My task this week is to build three Elasticsearch servers and get them running with three versions of SuiteCRM (7.12 (x2) and 8.0). What a hassle…

The update to Elasticsearch will be necessary in the immediate future (7.12 or 8.0) so you might as well bite the bullet and configure it as best you can. You’ll have no choice soon anyway.

Have you looked into the technicals on how to search Elasticsearch with special characters? There are a lot of languages using special characters so this must be a common issue.

isaacmarco · 25 January 2022 10:56

@CRMSigns, thank you for your answer

I have replay my migration from 7.6.7 now to 7.11.23 because in this version elasticsearh was introduced and can be activated. The thing were better but the problems with the spanish special characters persist.

I found this PR https://github.com/salesagility/SuiteCRM/issues/7919 related with my problem. I have merged this commit https://github.com/Dillon-Brown/SuiteCRM/commit/7c6dc9488df1ef9b19294c4748929dd422bbd401 and now I’m able to get good results when searching in console with Robo but those results are ugly when making the same search in UI with the global search.

These are my results serachig for “Natalia Muñoz Saavedra” in console with Robo.

root@azucar:/var/www/sugarcrm# php vendor/bin/robo -vvv elastic:search "Natalia Muñoz Saavedra" 

Results
=======

Leads
-----

 * Natalia Muñoz Saavedra
 * Natalia Natalia
 * Angelika Natalia
 * Natalia Muñoz
 * Natalia Muñoz
 * Natalia Muñoz
 * Natalia Muñoz
 * Natalia Ramirez Saavedra
 * Natalia Martín Saavedra
 * Natalia Muñoz Muñoz
 * Natalia
 * Carlos Saavedra
 * Vanesa Saavedra
 * Estefanía Saavedra
 * Natalia  Muñoz Villalón
 * Natalia Muñoz Urdiales
 * Natalia Muñoz Fernández
 * natalia muñoz liñán
 * Natalia Muñoz Valdés
 * Natalia García Muñoz

                                                                                                                        
 [OK] Search performed in 29 ms

And that is what I get searching with the UI.

I have no idea about especial tuning in elasticsearch motor but I think the problems are in SuiteCRM. I think that multinational companies works with special characters on many languages at the same time and utf8 now support that and Basic Global Search deals well with this (but it is slow)… in fact my search results are good when searching with robo in console.

I didn’t make tests with SuiteCRM 8 because at the moment there’s no path to upgrade there from previous versions.

Mac-Rae · 26 January 2022 10:51

Hi @isaacmarco & @CRMSigns,

We’re working to fix an issue with special characters and basic search which should be resolved with the next minor release. I’ll drop an update once this release is available as getting confirmation from yourselves whether this issue is also fixed would be beneficial

dtosun · 1 February 2022 20:30

Hello,

I have this issue for Turkish character(ç,ğ,ş,ö). After installing 7.12.2, i could not search with Turkish character.

I hope it can be solve as soon as possible.

isaacmarco · 2 February 2022 09:40

@ dtosun, last week SuiteCRM 7.12.3 was released and this version resolves for me the problems with Global Search and special Spanish characters when configuring “Basic Search” in the Global Search.

At the moment the problem persist when enabling Elastic Search.

isaacmarco · 2 February 2022 09:50

By other hand review the release notes for 7.12.3 because an important bug related with UTF8 was solved and may be you could be required to run some additional steps in order to re-encode some data in your database.

pgr · 2 February 2022 10:23

Currently that procedure is broken and may cause data loss… everybody is advised to wait for the next version fixing this

dtosun · 3 February 2022 12:07

@isaacmarco , Your special Spanish characters problem was solved without run UTF8 repair tool? I don’t want to run it because it may cause data loss.

Mac-Rae · 7 February 2022 09:54

It should be at least improved if not fully resolved. Depends really on when data was created (Data created in or past Version 7.11.18 until 7.12.3 require the repair once it has been fixed). We have a high priority fix for the issue currently in progress.

isaacmarco · 8 February 2022 11:59

My upgrade was from 7.6.7 to 7.12.3 so we are not required to run the repair utf8 tool.

dtosun · 8 February 2022 12:34

I don’t think so. My version is 7.8, when upgrade it to 7.12.2, i couldn’t search special characters.

Mac-Rae · 8 February 2022 12:50

Yes, you will need to upgrade to 7.12.3 to fix this functionality. Any data created in 7.12.2 will not work when you upgrade to 7.12.3 ether, you must wait until the new data repair scripts have been made available. As said we have a high priority fix for the issue currently in progress.

isaacmarco · 8 February 2022 20:45

@ dtosun , if you upgrade from 7.8 to 7.12.3 you can safely create new leads or edit old leads, but with 7.12.2 every time you edit some lead with special characters, those characters are replaced by other similar but not the same you type with the keyboard and data is partially corrupted.

If you are with 7.12.2 in a production environment I suggest to upgrade to 7.12.3 in order to stop to get corrupted data and when the fix that @ Mac-Rae comments that is in progress you will be able to repair. In 7.12.3 you will be able to use the search with no corrupted leads, accounts…

dtosun · 9 February 2022 06:22

@isaacmarco , I’m seeing my special characters on my application. There are no problem at 7.12.2. I just don’t search them via filter or basic search. I will try it with 7.12.3.

dtosun · 9 February 2022 07:46

After upgrade to 7.12.3, my data and search are fine now. But when i add Target module to basic search, there is an error like this. After disable Target module in settings, it is working good. Anyone has this issue? It was working fine at 7.11.5

Wed Feb  9 10:43:28 2022 [1521265][e433d2f0-c23e-3ad2-bd4d-5ee727929421][FATAL] [ERROR] [SearchThrowableHandler] An unknown error has occurred while performing the search.
Wed Feb  9 10:43:28 2022 [1521265][e433d2f0-c23e-3ad2-bd4d-5ee727929421][FATAL] [ERROR] [SearchThrowableHandler] TypeError: Return value of SuiteCRM\Search\SearchResults::getRelatedId() must be of the type string, null returned in /var/www/crm/crm/lib/Search/SearchResults.php:227
Stack trace:
#0 /var/www/crm/crm/lib/Search/SearchResults.php(179): SuiteCRM\Search\SearchResults->getRelatedId()
#1 /var/www/crm/crm/lib/Search/SearchResults.php(162): SuiteCRM\Search\SearchResults->updateObjLinks()
#2 /var/www/crm/crm/lib/Search/SearchResults.php(145): SuiteCRM\Search\SearchResults->updateFieldDefLinks()
#3 /var/www/crm/crm/lib/Search/UI/SearchResultsController.php(132): SuiteCRM\Search\SearchResults->getHitsAsBeans()
#4 /var/www/crm/crm/lib/Search/SearchEngine.php(102): SuiteCRM\Search\UI\SearchResultsController->display()
#5 /var/www/crm/crm/lib/Search/SearchEngine.php(78): SuiteCRM\Search\SearchEngine->displayResults()
#6 /var/www/crm/crm/lib/Search/SearchWrapper.php(94): SuiteCRM\Search\SearchEngine->searchAndDisplay()
#7 /var/www/crm/crm/modules/Home/Search.php(50): SuiteCRM\Search\SearchWrapper::searchAndDisplay()
#8 /var/www/crm/crm/modules/Home/UnifiedSearch.php(41): require_once('/var/www/crm/cr...')
#9 /var/www/crm/crm/include/MVC/View/SugarView.php(823): include_once('/var/www/crm/cr...')
#10 /var/www/crm/crm/include/MVC/View/views/view.classic.php(72): SugarView->includeClassicFile()
#11 /var/www/crm/crm/include/MVC/View/SugarView.php(210): ViewClassic->display()
#12 /var/www/crm/crm/include/MVC/Controller/SugarController.php(432): SugarView->process()
#13 /var/www/crm/crm/include/MVC/Controller/SugarController.php(363): SugarController->processView()
#14 /var/www/crm/crm/include/MVC/SugarApplication.php(101): SugarController->execute()
#15 /var/www/crm/crm/index.php(52): SugarApplication->execute()
#16 {main}

Mac-Rae · 9 February 2022 09:52

I’ll take a look at this one, may be a bug we can fix

dtosun · 9 February 2022 10:21

Hi @Mac-Rae ,

When i try it with new installation of 7.12.3, it is working fine. But when i try it with upgrade from 7.11.15 to 7.12.3, it is not working.
It was working fine with 7.11.15. May be some files were not updated after upgrade.