Migrating from 7.14.4 to 8.6.1: database migrations fail with 1071 "Specified key was too long; max key length is 3072 bytes"

Description
I am trying to migrate from 7.14.4 to 8.6.1 following the
migration instructions

Everything goes well until the post upgrade command, where the following error is encountered:

SuiteCRM Finalize Upgrade
============

Running: legacy-post-upgrade
step: legacy-post-upgrade | status: done
Post Upgrade process complete
Running: add-new-config-entries
step: add-new-config-entries | status: done
No new entries to add to the config file
Running: metadata-merge
step: metadata-merge | status: done
Metadata successfully merged
Running: run-migrations
step: run-migrations | status: failed
Error running migrations

In ./logs/upgrade.log I get:

CRITICAL [console] Error thrown while running command "doctrine:migrations:migrate --no-interaction --allow-no-migration". Message: "An exception occurred while executing 'CREATE TABLE migration_versions (version VARCHAR(1024) NOT NULL, executed_at DATETIME DEFAULT NULL, execution_time INT DEFAULT NULL, PRIMARY KEY(version)) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ciENGINE = InnoDB':\n\nSQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 3072 bytes" ["exception" => Doctrine\DBAL\Exception\DriverException { …},"command" => "doctrine:migrations:migrate --no-interaction --allow-no-migration","message" => """ An exception occurred while executing 'CREATE TABLE migration_versions (version VARCHAR(1024) NOT NULL, executed_at DATETIME DEFAULT NULL, execution_time INT DEFAULT NULL, PRIMARY KEY(version)) DEFAULT CHARACTER SET utf8 COLLATEutf8_unicode_ci ENGINE = InnoDB':\n \n SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 3072 bytes """]

My understanding The error occurs when creating the migration_versions table, which has a primary key column version with a length of 1024 characters. This exceeds the default maximum key length of 3072 bytes (or 3072 characters if using a single-byte character set like latin1).

Since the table is using the utf8_unicode_ci character set, which can use up to 4 bytes per character, a 1024-character column can potentially exceed the 3072-byte limit for index keys.

Expected Behavior According to the MariaDB 10.11 documentation, the innodb_file_per_table and innodb_file_format=Barracuda settings are enabled by default, which should allow creating tables with index keys longer than 3072 bytes.

Actual Behavior Despite the default settings in MariaDB 10.11, the “Specified key was too long; max key length is 3072 bytes” error is encountered when creating the migration_versions table.

Environment

  • VPS
  • SuiteCRM in Docker, based on php8.2-apache
  • PHP 8.2
  • MariaDB in Docker, version: 10.11
  • SuiteCRM version: 7.14.4 / 8.6.1
  • Operating System: Linux

Attachments

  • MariaDB configuration file (my.cnf)
[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

[server]
character-set-client-handshake = FALSE
collation-server = utf8mb4_unicode_ci
init-connect = 'SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci'
character-set-server = utf8mb4
  • database configuration for SuiteCRM (config.php)
  'dbconfig' =>
  array (
    'db_host_name' => '[edited]',
    'db_host_instance' => '',
    'db_user_name' => '[edited]',
    'db_password' => '[edited]',
    'db_name' => '[edited]',
    'db_type' => 'mysql',
    'db_port' => '3306',
    'db_manager' => 'MysqliManager',
    'collation' => 'utf8mb4_unicode_ci',
    'charset' => 'utf8mb4',
  ),
  'dbconfigoption' =>
  array (
    'persistent' => true,
    'autofree' => false,
    'debug' => 0,
    'ssl' => false,
    'collation' => 'utf8mb4_unicode_ci',
    'charset' => 'utf8mb4',
  ),

Attempted Solutions

  • Upgraded MariaDB server to 11.1 (I know it is not in the compatibility matrix, but maybe it could have solved this specific problem) → same error occurs.

Additional Information

I have searched fairly extensively but similar errors seem to happen in different contexts with older database versions - with MariaDB 10.11 or up it should not happen.
Any ideas?

I hope this will work.

Change Character Set and Collation:

CREATE TABLE migration_versions (
    version VARCHAR(1024) NOT NULL,
    executed_at DATETIME DEFAULT NULL,
    execution_time INT DEFAULT NULL,
    PRIMARY KEY(version)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE = InnoDB;
1 Like

Executing it manually led to the same error with MariaDB.

So I decreased the length to 700:

CREATE TABLE migration_versions (
    version VARCHAR(700) NOT NULL,
    executed_at DATETIME DEFAULT NULL,
    execution_time INT DEFAULT NULL,
    PRIMARY KEY(version)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE = InnoDB;

It worked.

I then executed the post upgrade script again, it did not try to create the table again:


SuiteCRM Finalize Upgrade
============

Running: legacy-post-upgrade
step: legacy-post-upgrade | status: done
Post Upgrade process complete
Running: add-new-config-entries
step: add-new-config-entries | status: done
No new entries to add to the config file
Running: metadata-merge
step: metadata-merge | status: done
Metadata successfully merged
Running: run-migrations
Warning! SuiteCRM is now using angular 16.
Warning! 'defaultExt' and any extension that contains frontend changes will need to be rebuilt. For defaultExt you can build using `yarn run build:defaultExt`
Warning! 'defaultExt' has been disabled. It may need to be rebuilt.
Warning! Extensions other than 'defaultExt' will need the following files manually updated:
- extensions/your-extension/app/tsconfig.app.json
- extensions/your-extension/app/webpack.config.js
- extensions/your-extension/config/extension.php
step: run-migrations | status: done
Successfully run migrations
Running: finalize-clear-symfony-cache
step: finalize-clear-symfony-cache | status: done
Successfully cleared cache
Warning! Please re-set permissions after this upgrade step has completed

So this time it worked!
Thanks a lot.

I do not feel 100% comfortable creating the table myself, but at least for testing purposes, I’ll use this. Also, I am not certain how long the “version” field needs to be for it to work in the future.

After this, I cannot log in (“Error occurred while retrieving records”) but this is another battle.

1 Like

oh no! Do you more errors in the log file? You will find some work-around in this forum for this problem.

More complete feedback for those who may have similar problems:

  • If you use utbfmb4, maximum length for index fields (like version) is 768.
  • If you use utfmb3, it can remain at 1024.

I guess cutting down version to 768 should be ok, but you can also choose to use ut8mb3 just for this table, as I guess you will not need emojis in the near future for its contents.

Anyway, so far in the upgrade process, I have to handle this manually, and it is not 100% obvious.

1 Like