EDIT: I missed a crucial negative in this first sentence wrong so I edited it
I think you’re right, the PHP files DON’T need executable permission. But it is true that 99% of installations out there (including mine) have that bit set, that’s how the recommended permissions are proposed. So there might be something unexpected, or less tested, happening here. A simple way to test it is to add those permissions and see if the operation succeeds.
Watch out for execute permissions on directories - those are needed to traverse the directory.
Something like this could help:
find /path/to/base/dir -type d -exec chmod 2750 {} +
I tend to be very cautious following install guides with blanket permissions. PHP files shouldn’t need execute bit unless you want to run them directly as a script using the shebang to specify the interpreter.
root@suitecrm:/opt/apps/suitecrm# head -n1 vendor/bin/robo
#!/usr/bin/env php
If you aren’t regularly running these as scripts (e.g. from CLI rather than webserver) then there is no need for that shortcut and you can just specify the interpreter and the execute bit isn’t needed on the php script (e.g. php /path/to/no-execute-script.php.
I would guess the only time you would need execute permissions would be if you had a php script you ran (e.g. robo) that was trying to execute another php script (not include) as a separate process by invoking the script name directly without specifying the php interpreter as the binary before the script path.
As I mentioned earlier, I generally use find to apply permissions because files and directories need different treatment. As you noted directories DO need execute permission for listing/traversal. Our SuiteCRM setup currently uses 660 on files and 2770 on directories and this works fine. By the way the reason for permissive group permissions and the setgid on directories is because we have SFTP users that are added to the group and need to be able to read/write/create files. Files created need to belong to same group so webserver can still read them and vice versa. Often we use a different group than www-data something like suitecrm-admins which will include any suitecrm SFTP users as well as the webserver user. On fully provisioned, no manual editing setups, then webserver user is the only one that needs any access.
I think we were posting at the same time. This turns out to have nothing to do with permissions and is a bug with path resolution for autoload.php in the robo script.
Just answering my own questions from earlier for anyone else with this problem.
installing composer package is not necessary if you like to keep unneeded packages off your production systems.
Running composer install is NOT necessary either, the error message is misleading. You just need to fix the paths in robo script so it can find autoload.php.
Installing the intl php module is apparently also not necessary. Yes it is needed if you want to run composer install but it is apparently not needed for install or upgrades of the system.
When you first install from zip, the installer php script also checks for missing modules and does not complain about missing intl php module…so it is apparently not needed.
I edited my post above, I meant to write that execute bit should NOT be needed, that’s why I said “I think you’re right”
Anyway, remember that SuiteCRM does a lot of stuff from CLI: all the Scheduler Jobs, so that’s Workflows, search indexing, notifications, email campaigns…
Thanks for your notes on Composer, I also think that some of these error messages are red herrings, it’s supposed to work in a much easier way.
Okay. So far I haven’t seen any errors…depends on how they are being run whether they need the execute bit or not. If there was a list of X,Y,Z are CLI apps rather than included scripts and need +x permission because they are run directly as sub-processes without invoking the php binary first…that would be great. It doesn’t give me a good feeling to blanket execute permissions across all php files “just in case”.
In terms of the bug(s)…
sounds like the back/cancel button issue in the UI has been noted already.
What is the best way to report the path issue with robo so it can be fixed?
Any thoughts on why I seem to be the only one running into this? Some breaking change to how __DIR__ works in PHP7.2 on Ubuntu 20? Or just no one using the command line upgrader?
Okay. If I run into errors and figure out that specific php files do need executable permission I will update the docs. Until then, I’ll stick with 660 on the php files since it seems to work fine.
Okay well there isn’t much room for interpretation of code. There are 2 search paths to autoload.php configured in robo and they both use __DIR__ to construct the paths. The paths constructed are both wrong. So either the configured paths in robo are wrong (bug) or something about how PHP defines __DIR__ has changed (also bug) or autoload.php is in the wrong location (also bug). I can’t imagine what other underlying condition could possibly have any influence on this problem since we are down to a couple lines of code that are quite easy to understand. Once I fix the path to autoload.php in robo the installer works fine…so this is not a permissions issue.
Would be nice to get this fixed so I don’t have to patch robo every time I want to do a security update of SuiteCRM. I can open a pull request but it will be very naive because I don’t know any history or context about why those 2 paths are there in the first place and whether they are still relevant in any valid scenarios. The PR would either delete one /../ from the second search path or add a third search path as in the example I posted above. But as I said, no idea how this was intended to work, so given this is a small change and a bug rather than an enhancement…probably better to report it and leave it with a maintainer that actually has the context on how this is supposed to work.
In terms of where this lives in git…that I don’t know as I am not a PHP dev (don’t know build processes or toolchains) and from a quick search it appears the robo file is generated or at least is not part of the main SuiteCRM repo.
All the stuff under vendor directory is 3rd party, it is not stored in our git.
The basic process SalesAgility uses to build the upgrade packages is to grab our code from Github, then pull all that vendor code in with composer, and zips it along with a few additional scripts (if needed).
So I’m not entirely sure why robo code would be expecting something that it can’t find in your installation (or perhaps in all SuiteCRM installations, but I don’t think that is the case). Unless our upgrade script changes working directory and confuses Robo?.. but that seems impossible, the problem is right at the beginning of Robo execution…
Okay. I just searched “php robo” and found what looks like the repo you are using this from.
That project doesn’t provide autoload.php. Apparently autoload.php is generated by composer. I don’t know which composer on which project whether it is SuiteCRM or the robo project. Nevertheless it ends up in the wrong place. Either because the composer build puts it in the wrong place, or because it is put in the wrong place when SalesAgility zips up the vendor stuff.
As I said this is the zip from you guys and I haven’t modified it…so pretty sure command line upgrade is broken for everyone.
It is easy enough to check. The code isn’t doing much here. Just download a zip, extract it, check the vendors folder. Is autoload.php at ./vendors/autoload.php? If yes then the command line upgrade is broken for you too. Try running a command line upgrade on a zip (non-git) install following the docs.
The paths in the upstream robo repo I linked match what I see in my zip from you guys. So I guess autoload.php is being put in the wrong place by whatever composer builds it, or the vendor folder building process.
As you said you don’t think many people are using command line upgrade…so perhaps this is just broken and no one noticed.
You are right, there is no ./vendor/ folder in the normal install zip.
So this must have been created when I tried to first run the Upgrade in the UI upgrade wizard (which failed, was broken).
So I guess to reproduce this you have to go half-way through a UI upgrade (upload upgrade package and then bail) and then cancel and then check the vendors folder. I will need to do a fresh install to try to repo again…but part of the reason I am confused in the following…
The command line upgrade instructions say to run ./vendor/bin/robo ...… but as you pointed out, the vendor folder isn’t there on a new install…so you can’t do a command line upgrade. It never occurred to me that the vendor folder might not be there on a fresh install since it was there when I was following the command line upgrade instructions the first time (after the UI upgrade failed).
Different question then… is it possible to run a command line upgrade from a new install or only use the command line upgrade after you have done at least one UI upgrade? Once I understand this better I will happily update the docs to reflect the reality. Thanks!
Thank you for the explanation that the vendor folder is generated by running the installer, that gives me a more specific place to look in the source tree for this problem.
I have this problem on two different (independent boxes) that were both fresh installs from zip from 7.11.18 in the last months. One of them I had a bunch of UX/db problems with the installer (error messages not surfacing). The second one didn’t have any issues as I knew the issues from the first time around. Point is there wasn’t anything weird that happened during the install. I just realized it can’t related to the broken UI upgrade because the second box I didn’t try the UI upgrade…I went straight for the command line updater and it had the same issue.
So I would be surprised if this was something weird about my installs as they are just extracted zips with the installer run. Nevertheless I guess I need to try and track the problem down in your installer or make a screencap repro or something if you can’t reproduce this on your end.
What I did see already searching the SuiteCRM repo for “autoload.php” is that there are quite a few references to that file being located at ./vendor/autoload.php…which is indeed where it is located…but that doesn’t work with the autoload search paths in the third party robo script. So I suspect that the autoload.php is in the expected place from SuiteCRM’s point of view but there is some incompatibility with where SuiteCRM wants that file to be and where the third party robo script wants that to be.
Maybe you could look at one of your installs for comparison? Is autoload.php located in ./vendor/? If so does running a command line upgrade work for you? You don’t need to run an actual update (give a non-existent upgrade zip name) to see if robo finds autoload or errors out with the message about running composer install.
I understand if you don’t have time to try and repro on your end. In that case I will just try to track down the installer code.
But at this point…
I am pretty sure that ./vendor/autoload.php is in the place SuiteCRM expects it to be
due to multiple references to that location in the SuiteCRM git tree
I am very confident that robo is looking in the wrong paths to find ./vendor/autoload.php
robo git repo matches what I have on disk
fixing the paths in robo to point to where SuiteCRM has autoload.php resolves the issue
So the more interesting question is what else is going on (different) that causes me to experience this and others not. Perhaps…
how robo is being run
e.g. I don’t know what “phar” is…but it results in a different search approach for autoload.php.
composer install related stuff
maybe people running composer end up generating an additional autoload.php somewhere that robo can find it?
I’m assuming others use the command line upgrader and don’t experience this issue and that you can’t reproduce/verify this issue on your own suitecrm installs.
When I say download the installer zip, I mean from here:
That’s the one that is built like I said, and has the vendor directory. The installer does not build the vendor directory, it’s the internal process in SalesAgility when building the installer package that does that.
You have been talking all along about “the zip” as the Github release pacakge. There are no instructions or recommendations that I know of which describe installing SuiteCRM from that. If you grab it from there you will surely have to run composer install --no-dev before you start running anything.
It sounds like I am frustrating you. Sorry about that, I am also just looking for clarity here.
You are right, in my “check if vendor folder is in zip” I used the github release.
I wasn’t aware there was a different build for github and what salesagility publishes on their site.
I definitely don’t do composer install during our deploy, so the zip we deployed on the 2 servers must have been downloaded from sales agility (because it works fine without running composer).
So let’s forget my previous comment about vendor directory not being in the zip because I was looking at the wrong zip.
Looking at the correct zip…
Checking zip contents of non-github zip
root@suitecrm:/tmp/SuiteCRM-Test# wget "https://suitecrm.com/files/162/SuiteCRM-7.11/548/SuiteCRM-7.11.20.zip"
root@suitecrm:/tmp/SuiteCRM-Test# unzip -q SuiteCRM-7.11.20.zip
root@suitecrm:/tmp/SuiteCRM-Test# ls -l SuiteCRM-7.11.20/autoload.php
ls: cannot access 'SuiteCRM-7.11.20/autoload.php': No such file or directory
root@suitecrm:/tmp/SuiteCRM-Test# ls -l SuiteCRM-7.11.20/vendor/autoload.php
-rw-r--r-- 1 root root 178 Jun 1 14:11 SuiteCRM-7.11.20/vendor/autoload.php
root@suitecrm:/tmp/SuiteCRM-Test# tail -n +18 SuiteCRM-7.11.20/vendor/bin/robo | head -n 5
// Non-phar autoloader paths
$candidates = [
__DIR__.'/vendor/autoload.php',
__DIR__.'/../../autoload.php',
];
root@suitecrm:/tmp/SuiteCRM-Test#
(collapsed above)
vendor directory is there.
autoload.php lives at ./vendor/autoload.php
robo (./vendor/bin/robo) looks for autoload.php at ./vendor/bin/vendor/autoload.php and ./autoload.php
…and won’t find it.
resulting in the title error of this post when trying to run a commandline upgrade using robo per the docs.
Manually editing the autoload.php search paths in robo to match the structure of the zip (ie so it can find autoload.php) resolves the issue and the upgrade then proceeds without issue.
Am I missing something here? Not trying to annoy you, was really just trying to be helpful in tracking down the root of this issue.
Don’t worry, I’m not annoyed, and didn’t mean to sound annoyed Sorry!
After 14.5k posts here, I think I am doubly vaccinated against all forms of ignorance, misunderstandings etc. They are the daily bread of Forum life. And you’re waaay above average in competence diagnosing and clarity and politeness writing.
About our issue here, I also agree it’s worth getting to the bottom of this.
I see a difference between your installation and mine. In mine, robo lives in /vendor/consolidation/robo. That’s a directory name, so the PHP file is /vendor/consolidation/robo/robo. This would work with the autoload.php path it’s expecting.
Maybe your Robo (or composer?) is an older version, or was installed differently? I think it wasn’t always like this, but the standard now is to always use a company name as a folder before using a product name…
In the example from my previous post (wget non-github zip, extract, inspect files that were in zip) you can see robo lives in ./vendor/bin/robo. This is the latest SuiteCRM (non-github) release zip. I haven’t done anything to it except extract the zip. I have not run the install script either.
Based on your reply, I checked…it ALSO lives at the path you mentioned…
root@suitecrm:/tmp/SuiteCRM-Test# ls -l SuiteCRM-7.11.20/vendor/consolidation/robo/robo
-rw-r--r-- 1 root root 1630 Oct 11 2020 SuiteCRM-7.11.20/vendor/consolidation/robo/robo
So maybe this is just an issue of the docs instructing a user to run the wrong robo when doing a command line upgrade?
Download the required upgrade pack and place it in the folder with the installed system
From the folder with the installed system, run the command ./vendor/bin/robo upgrade:suite [upgradeZipFile] [logFile] [pathToSuiteCRMInstance] [adminUser] …
We’ve gone around in circles a bit here, so to sum up…
that robo script searches for autoload.php on paths where it won’t find it
which results in the autoload.php not found maybe run composer install (paraphrased) error.
fixing the search path for autoload.php in robo resolves the error (composer install is NOT needed) and the upgrade works. So the title error of this thread is really just about the robo script not finding the autoload.php.
this is based on the latest SuiteCRM (not github) release zip code (download, extract and inspecting the zip file).
So I guess either the
docs are wrong (e.g. we are told to run the wrong robo) or
./vendor/autoload.php is in the wrong place (although it might be in the right place for other parts of SuiteCRM that expect it there)
I am not sure how to sum up, but I’d say the Docs should probably be updated to say ./vendor/consolidation/robo/robo
We should try to find out how come we have a robo PHP file in /vendor/bin.
@Dillon-Brown do you know how that gets in there? Do our packaging scripts put it there, or is it composer install? Any reason to have 2 robos, one of them in a path where it doesn’t work properly?
As there have not been any further replies from the SalesAgility side, I’ve created a github issue to track this since I think we both agree this is broken per the docs and isn’t a problem between the keyboard and chair or something “special” about my vanilla Ubuntu 20 server. There are about 1.1k github issues on the SalesAgility github project!!! So not holding out hope any PHP dev will actually look at this… but at least it will be tracked as an open issue for others that run into this.
Sorry!