Please help save my server

I’m not exactly sure where to start with this. So I will start from the beginning.

When we first started with SuiteCRM, we decided to go with the appliance for our virtual environment. And things have been great. Yesterday we started to get reports people couldn’t log in. They were given the error, “You have been logged out because your session has expired.”

Now we have experienced this before (but not for a while) and we would just reboot the server. Then everything was fine. But this time it did not work.

I am able to log into the administrator account on the Univention control panel. But we can’t get into SuiteCRM. The Univention control panel alerted us that our running version was out of date.

We at that point decided to just update the server thinking it couldn’t get any worse. Well, we instantly got errors stating there was no space left in a partition. I decided to SSH into the server and run “df -h”, but nothing was out of space. After some searching I ran “df -i” and that was when we seen we were all out of inodes in the directory.

We chased down where all the inodes were being used and there was over 6 million files in “/var/lib/docker/overlay2/6a0af7b9c4368e9fbb951c42c09d29d8fd2ad9ae3358f8600fbd4ce10ffd1ad0/diff/var/lib/php/sessions”. So we deleted the file in it’s entirety and recreated it with the same permissions.

We checked the inodes again and all looked good. So we started the package upgrades. Now it is stuck in an endless loop and not failing out. This is what the log says…

"Starting univention-upgrade. Current UCS version is 4.4-7 errata898

Checking for package updates: found
Warning: cannot update /var/lib/univention-updater/univention-upgrade.status.new
Warning: cannot update /var/lib/univention-updater/univention-upgrade.status
Please rerun command without --check argument to install.

Starting univention-upgrade. Current UCS version is 4.4-7 errata898

Checking for package updates: found
Please rerun command without --check argument to install."

Any help would be greatly appreciated. Thanks in advance.

Hi, welcome to the Community :tada: !

Sorry to hear about your troubles. I’ve seen that session directory problem before. But it seems you fixed that now.

Your current problem is, I believe, a Univention problem, which I don’t know much about. I wonder what is the best place to get support on that, but it’s probably not here, since I don’t people discuss that much here…

If you still have a backup of the old appliance, the one that ran out of inodes, I suggest saving that one by deleting the session files (some care is needed for this, not all rm commands will do the job).

Thank you for your response. And thank you for the welcoming.

We do have backups we could restore to. I was saving that for a last ditch effort. I would love to solve this though so if it happens to anyone else there would be an answer. But that answer may be just restoring the server to a couple days ago.

If you choose to delete the files this is a good start:

Docker containers doesn’t have cron, so /usr/lib/php/sessionclean script doesn’t execute.
You may execute the script from the host machine, for every running php container

So I managed to update Univention to 4.4-9 errata1371. It won’t go to 5.0 because of SuiteCRM as stated here in the logs…

The update to 5.0 is currently not possible,
because the following Apps are not available for UCS 5.0:

  • SuiteCRM

BUT I think I have 2 issues here. One with UCS which is now seemingly stable. The other is when anyone goes to log into SuiteCRM, they get this error…

“You have been logged out because your session has expired.”

Anyone deal with this error before?

There are many things that cause that error, but one of them is exactly any problem with the session directory. The directory that you define in your php configuration must exist and be writable by the web server.

Your root directory could be to full.