Database Seeding

Hi! I was hoping I could start a conversation here about improving the database seeding system in SuiteCRM.

Right now, the only seeding system is creating static data from a handful of SQL files.

I imagine it’d work like this:

  • You create seed code in include/seeds/ (or some other directory, whatever people think would be the logical place for this.)
  • ./vendor/bin/robo db:seed would run all the files in that directory, and seed the database with data generated by the seed code.
  • You could also run something like ./vendor/bin/robo db:seed --only=Accounts,Contacts that would only run Accounts and Contacts seeders, for example.

You can see Laravel’s Seeding docs for something similar to what I’m imagining.

The reason this is useful is because it helps easily create fake data for development, without needing to pull the data in from production or create it manually yourself.

We’d probably want to have factories (Laravel) that we can also use in tests?

# Example of a Laravel Factory
use Faker\Generator as Faker;

$factory->define(App\User::class, function (Faker $faker) {
    return [
        'name' => $faker->name,
        'email' => $faker->unique()->safeEmail,
        'email_verified_at' => now(),
        'password' => '$2y$10$92IXUNpkjO0rOQ5byMi.Ye4oKoEa3Ro9llC/.og/at2.uheWG/igi', // password
    ];
});
# Then in a test:
public function testUser()
{
    $user = factory(App\User::class)->create();
    # Test things the user can do.
}

# Or in a seeder:
public function seed()
{
    # Make 5 users.
    $user = factory(App\User::class, 5)->create();
}

There are also states, attributes, and other things, but that’s not super important for a first draft of this system.

We already have some things in tests that do this, but a lot of them do it via the frontend and the system isn’t standardized. It also doesn’t really use random data like it probably should.

2 Likes

Hi @connorshea !!

I would love to have something like this in SuiteCRM!
Almost every day I see myself creating Fake data in SuiteCRM manually :frowning_face:

If you want, we can work together in the development.

1 Like

My idea would be to make the factories as fast as possible, with the ability to add attributes to the created bean when defining it.

class UserFactory extends BaseFactory {
  // Default attributes should include as few attributes for the bean as
  // possible, only as much as is needed to create the bean. The developer can
  // then either pass more attributes to define as needed, or we can create
  // 'traits' that the developer applies, e.g. 'adminUser'.
  static default_attributes = [
    'name' => $this->faker->name(),
    'email' => $this->faker->unique()->safeEmail()
  ];

  function define($attributes = []) {
    $user = BeanFactory::newBean('User');
    
    // Magic code here to automatically merge the default attributes and any
    // attributes sent via `define()`. Then, add them as properties
    // on the user bean:
    // $user->name = default_attributes['name'];
    // $user->email = default_attributes['email'];

    // Then save and return the user.
    $user->save();
    return $user;
  }
}

Here’s my current spike on the idea, although it doesn’t quite work yet: https://github.com/connorshea/SuiteCRM/tree/seed-factory-spike

This system is inspired pretty heavily by Ruby’s FactoryBot, which looks like this:

factory :user do
  username { "Foo" }
end

factory :story do
  title { "My awesome story" }
  user

  trait :published do
    published { true }
  end

  trait :unpublished do
    published { false }
  end

  trait :week_long_publishing do
    start_at { 1.week.ago }
    end_at { Time.now }
  end

  trait :month_long_publishing do
    start_at { 1.month.ago }
    end_at { Time.now }
  end

  factory :week_long_published_story,    traits: [:published, :week_long_publishing]
  factory :month_long_published_story,   traits: [:published, :month_long_publishing]
  factory :week_long_unpublished_story,  traits: [:unpublished, :week_long_publishing]
  factory :month_long_unpublished_story, traits: [:unpublished, :month_long_publishing]
end

create(:story) #=> story with a title and a user.
create(:story, traits: [:published]) #=> story with a title and user that has been published.

I would like to think in the user cases:

For example:

  • $ ./vendor/bin/robo db:seed <- This will seed the database with as much data as when we install SuiteCRM with demo data?

  • $ ./vendor/bin/robo db:seed --only=Accounts,Contacts <- The command will create Accounts, Contacts related to the seeded Accounts and Contacts not related to any Account? What about the Users? Should we seed Users, Roles and SecurityGroups always no matter what Module we are seeding?

I’ve opened a draft PR for the seeder/factories :slight_smile: https://github.com/salesagility/SuiteCRM/pull/8465

1 Like

Right now there aren’t enough factories built to do this, but once we add more factories, we should be able to replace the demo data system with it.

That said, I think populating the DB with the demo data should be done via a separate command. The db:seed command should be used to seed a pretty good amount of data, enough for developing on.

I’m not sure about this one, I think it should only seed as much as is needed to create valid Accounts and Contacts.