So, through some changes in the way WHM/CPanel works, my backups filled all of my hard drive space. Just for information, the way we diagnosed this, was that we couldn’t login because new PHP Sessions could not be created. Anyhow, I deleted the excess backups, and everything started working, except for the following…
The search bar on the top of every window doesn’t work anymore. I call this “OmniSearch”. And I get the following errors in the error log.
Thu Jan 5 22:00:05 2017 [3][1][ERROR] Unsupported segments file format
Thu Jan 5 22:00:05 2017 [3][1][FATAL] Job 1cdae66e-5a0e-a637-dd3c-586f32ed6885 (Perform Lucene Index) failed in CRON run
Thu Jan 5 22:01:02 2017 [3][1][FATAL] Job 227d1edd-dbfb-d37f-421c-586f3293d582 (Optimise AOD Index) failed in CRON run
I believe it is possible all I need to do is whipe out the index and allow it to reindex, but I’m not sure how to do this. My assumption is that space ran out during the index process and corrupted files.
Interestingly, searches within modules still run fine.
If you have ideas, please share. I will be working on this today, and will provide updates on progress.
Though not an expert in Lucene, reading about it seems that people who usually come to this issue remove the corrupted segments_N file that have 0 bytes and allow the system to re-index. These can be found in /modules/AOD_Index/Index/Index/
I copied the files from an old backup (October) to the AOD_Index/Index/Index folder, and search results are now working. The index is out of date, but I’m HOPEFUL that it will update overnight?
OK, so I changed the scheduler to run the “Perform Lucene Index” job. Alas that didn’t seem to resolve the issue (there are still items that don’t show up as search results). Any suggestions?
Is the Perform Lucene Index job still running? You’ve used an older index then it will need to re-index so I suspect it won’t be a quick thing considering that OP from that thread stated that it took him 3 days (or was a week) to re-index everything. So I suspect it will just required a little time to do so. As you said, perhaps leave it over night and see if the index file has grown.
Also, I just saw you original post. Indicating I should delete all records in aod_index and aod_indexevent. I did not do this. And I’m a little hessitant to. It makes sense that it could be the problem, however, if the database is indicating that things have been indexed that aren’t actually included in the index. I’m also a little nervous to do this because I didn’t wipeout the index directory, rather I replaced the files with old files. I’m not sure how the index works, but I would hate to have duplicate entries.
Quite honestly I am not 100% sure how to force lucene to re-index without deleting everything and allowing the system to catch up on itself like the OP said in his post. Is that a possibility on your end?
Just to bring in my experience with this, I have also had to rebuild AOD_Index and I also think it has to be done completely. I believe the cron job only updates changes. Since you started with files from some time back, now there is no way for the indexer to know what is missing.
Of course you should have dumps of those tables before deleting them, so you have a path back to safety (or at least, to where you are now).
