Enhanced seeding system

[DenuxPlays]

Hey,

since I created two prs regarding to improve the seeding system and a few questions came up I wanted to focus my motivation here.

My plan is to improve the seeding system to have something like symfony fixtures.
Currently it is only possible to seed via files which will fail if you need to generate data/ids or execute other code somehow.

So my first pr is to allow the execution of rust code.
My second pr was to pass AppContext to the seed function instead of just the db to allow more functionality to the user.

Further plans are:

• auto seed feature
• seed by env (for example dont seed dev data in production)
• seed by tags

This would immensely improve the dx and the whole fixtures/seeding feature of loco.
But it would also introduce a few breaking changes (change the file format of the seeding file to add the ability to add tags and env requirements and ofcourse the change from the DatabaseConnection to AppContext)

Ofcourse would be available to create prs and maintain the features in the future.

And ofcourse everything can be implemented by using the hooks but I think more users can take advantage of it which is why I am proposing it as a new feature.

CC: @kaplanelad

Ref issues:

• Auto seed #1062
• seed function in Hook should get AppContext as an argument. #1060
• Allow fixtures to execute rust code #972

Ref prs:

• Added ability to seed via rust code #1061
• Fixtures use AppContext instead of DatabaseConnection #1063

[DenuxPlays]

Key difference to migrations:

Besides the features I want to implement there is one other key difference:

migrations are keept inside another folder/crate which does not allow me to access my custom code inside them so it only makes them useful for basic schema migration.

In my case:
I would need to copy my whole snowflake generator into the migration crate and I hate code duplication.

[DenuxPlays]

Had you both time to look at this?
@kaplanelad
@jondot

[kaplanelad]

@DenuxPlays
I've shared my thoughts in both the issue and the PR.

I'm looking forward to hearing more feedback from the community here.

[Ghost-str]

Perhaps it’s worth using trait Environment instead of a hard-coded enum with test, dev, prod environment.
That trait should look like this:

pub trait Environment {
     fn get_name() -> &'static str;
     fn is_allow_seed() -> bool;
     fn is_allow_truncate() -> bool;
     async fn load_config() -> Result<Config, Error>;
     async fn seed() -> Result<(), Error>;
     async fn truncate() -> Result<(), Error>;
     async fn logger_init() -> Result<(), Error>;
}

we can use this for tests too, to reproduce some application environment

I feel like this will solve a bunch of different problems

[Ghost-str]

By design, you cannot run seeds in a production environment.
To initialize some data in the production environment, you need to use safe migration while considering the case of data rollback.
By design seeds should not be dynamic and with their help some kind of standard environment should be created to save employee time

[DenuxPlays]

Why shouldn't seeds be not dynamic?
Every other framework let me do everything in seeds.

Also we cannot use migrations as I do not have access to my models let alone to any custom as it is in a separate crate.

And why should seeds not be run in production envoirements?
What when I have a list of predefined system tags?
Insert them manually when I set up my stage or production server?

Then what should I do?

[Ghost-str]

What when I have a list of predefined system tags?

create migration or create custom task

And why should seeds not be run in production envoirements?

because it is a dangerous action, seeds do not have a safe mechanism to return everything as it was in case of any problems. And only the developer knows how to roll back the action

Also we cannot use migrations as I do not have access to my models let alone to any custom as it is in a separate crate.

A very strange mechanism, but how do you create tables for entities from another crate?

[DenuxPlays]

Again Migration I cannot use (See the base template. Migrations are a seperate crate)
Tasks would work but would Require a lot of extra Code and at this Point I could just build a normal lib

Yeah but what Happens when I the a Static seed doesnt work nothing is done to reset the System.

I do it via an sql file.
But I would use it anyway because SeaORM doesnt have the features I Need.

The best way would be to have the migrations inside the normal src folder and also give migrations Access to the AppContext

[DenuxPlays]

And again seeding is being used to prepare the database.
If the seeding fails there is no need for recovery as there is nothing to recover.
Recovery would always be a new installation.

[kaplanelad]

I'll give you my take from all your recommended features:

auto seed

In many applications (most of them), the auto seed feature is used to initially populate the database with default data during development. This feature aims to provide a good starting point for the application, ensuring that critical data is present right from the beginning.

Automatically seeding the database every time the server is reset can lead to several problems, such as overwriting important production data (in a new feature that you working), mixing development and production environments, by applying by mistake local data to production. This can create inconsistencies, reduce control, and increase the risk of data loss, particularly in sensitive or high-traffic applications.

That being said, if your application requires a different behavior, you can customize this by implementing your own approach in the before_run phase. Here’s an example to guide you:

 async fn before_run(app_context: &AppContext) -> Result<()> {
            // Option 1
            Self::seed(
                &app_context.db,
                PathBuf::from("src").join("fixtures").as_path(),
            )
            .await?;

        // Option 2
        // Call any custom seeding function that you want

        Ok(())
    }

seed by env (for example don't seed dev data in production)

As shown in the example above, within the before_run function, you have access to the app context. You can check the app_context.environment to determine the current environment, and then call the seed function with a different folder path accordingly:

let seed_path = if app_context.environment == Environment::Development {
PathBuf::from("default")
} else {
PathBuf::from("production")
}

  Self::seed(
                &app_context.db,
                PathBuf::from("src").join("fixtures").join(seed_path).as_path(),
            )
            .await?;

seed by tags

You can create a task named seed that accepts parameters indicating the type of seed you wish to apply. This allows for more granular control over what gets seeded.

Summary

From my perspective, seeding data is most relevant for development or deploying dynamic applications that require initial data. It seems you have a very specific scenario. Loco allows you to handle this using the before_run phase (although it may differ from other frameworks you might be familiar with, the flexibility is there).

As a maintainer of Loco, one of my goals is also to help users avoid mistakes—seeding data can be a risky action if not handled carefully.

Regarding #1060, I have created a PR that adds the app context in #1158. I don’t see any reason not to include it.