Does suitecrm store any data in files (uploads, attachments, etc) or is all data stored in db? - backups

đź’¬ General Discussion
1

Howdy.

We are still in testing phase with SuiteCRM.
Part of that is ensuring we have a sane way to manage auto deploys (done) and auto-backups.

Obviously I need to backup the db. No issue there.
My question is about any “data dirs” on disk (attachments, uploads, etc if these are stored in files rather than in the db).

Every discourse or blog post I have seen on the subject is talking about backing up the whole web-dir…ie the application dir.

I don’t want to backup the application dir because any restore I do will be on a fresh/new deployment… ie the application code will be freshly deployed.

So I only want to backup/restore any “data dirs” on disk.

If there are none…great, a db backup is all I need.
If there are specific data/upload dirs within the application dir… can someone give me a list?
Related question: Is it possible to configure these data dir(s) to live on a custom path(s)?

Dear plugin spammers/advertisers… please don’t post here about your backup plugins as I have no interest in plugin-based backup solutions.

2

@mattp

:joy::+1:t2: - about spammers.

About work.
You can copy files:

  • from data/upload. There are files loading in SuiteCRM. Their names have “id” as in database maybe other symbols as addition .
  • config_override.php this file rewrite system parameters.
    It’s by default.
3

@p.konetskiy are you sure you mean data/upload ? :thinking:

I would add

  • the entire custom directory, although you might want to exclude the files automatically generated by a QR&R, and then include a QR&R operation in your deploy script.

  • the upload directory, excluding upload/upgrades which is bulky and unnecessary

  • there are a few Studio or Module builder operations that will create custom modules in the root modules directory (not just in custom/modules)

So, it’s a bit messy, but definitely possible. I wouldn’t go about changing directories if I were you, it will get quite complicated.

P.S - are you considering deploying from git repository? There are a few advantages to that approach, to manage your own customizations and also to be able to pull fixes from GitHub. Since there are many valuable unmerged PR’s, this is sometimes convenient.

4

@pgr
I understand that @mattp want backup data only. Other directories are parts of application from system or custom areas.

5

Thanks @p.konetskiy and @pgr for the fast reply and detailed info.

Regarding config_override.php is this file generated by the installer and necessary for a restore or would this be a config file I provided that obviously I would need a copy of?

@pgr It does sound quite messy as it seems there is a lot of generated/necessary “state” files mixed in with the base application (in addition to the UGC uploads)…not very well defined exactly where the important “state” files are…

  • upload but not upload/upgrades
  • custom but not QR&R files
  • modules…some things…not clear which because some studio/module builders create things here instead of in custom/modules

It would clean things up quite a bit if all user and system generated (modules) content was able to be segmented off into a configurable DATA_DIR and CACHE_DIR to make “clean” deployments easier. CACHE_DIR could be used for any caching, update files, etc and is not necessary for backups/restore but partitions off that type of data from both the APP_DIR and DATA_DIRs.

The way we normally do this is by passing env with the relevant directory locations to the app when it is run. Of course we take care of creating the directories and making sure the app’s user has appropriate access/perms. In the php case we aren’t running an app but the webserver hands php files off to a “plugin” for processing (why we don’t need execute permission on the php files)… so it would need to be specified with config file for the php app instead…but should still be possible I would think without too much issue. Default directory location(s) could live in the APP_DIR root of course.

I understand your point about the git approach. I will keep that in mind as an option.

Generally I try to do the following:

  • only backup “state” and not application code. state includes database and “DATA_DIR” stuff.
  • backups contain current version of application code

Restores always restore with a fresh install of the same version of the application as the backup was made from to avoid data schema migration problems.

Restore just involves deploy of:

  • fresh application code (same version of app as backup)
  • database
  • DATA_DIR
  • config file(s)

This approach allows users to customize all they want because any “state” is either in the db or DATA_DIR and both are backed up and easily restorable.

Backups are well defined and restores can be done with “clean” application code…ie if your server gets hacked you know you are restoring with clean application code on the rebuild (plus the backed-up state).

Maintaining (pulling in changes) and deploying form our own git fork doesn’t really solve the problem for me because I don’t want to maintain state for a specific deployment in git with the application code…for instance if I want to deploy both staging and production from the same code-base and have different “state” for each.

But okay, that is the status quo and the exact info I was looking for. Just a bit disappointed that it is so complicated to extract the state for backup from the rest of the application code. :slight_smile:

6

The config_override.php is config of application which you can change manually. The file generated by the installer is config.php

If your clients will be have access to Admin you can NOT do backup data only. You should remember about other directories as wrote @pgr.

7

Well when I said “DATA” what I really meant was application “STATE”. So that can be data (e.g. user generated data) but also things like custom workflows and things that are necessary to re-deploy the app with fresh application code (same version as backup) with same state.

So it sounds like because the generated code/state is quite “tangled” with the main deployed application code, I don’t really have much option other than to backup the entire APP_DIR. This just means that I can’t do a “clean” deploy easily.

For instance when troubleshooting a problem I will often start editing the live application adding debug statements in random places. Once the problem is resolved I usually re-deploy with clean application code. When DATA/STATE dirs are able to be separated from the main application code…this is very easy to do…I don’t even need to restore a backup. :slight_smile:

Anyway, I understand this isn’t supported or easy to manage with the way Suite/Sugar was built…so I’ll leave this topic alone.

I appreciate you guys answering with detailed info and advice. Have a great day!

8

I agree this could be way more tidy than it is, but it’s not that complicated.

I never bother about the root modules dir, I don’t do anything there. So I just save upload and custom. The other considerations are just optimizations to save space in the backups.

1 Like