RS DB Seeder

RS DB Seeder makes it easy to populate database tables for your tests.

• Motivation
• Getting Started
  • DB Adapter
  • Factories
• Usage
• API
  • Build
  • Insert
    • References
    • Custom insert implementation
• Clean up
• Advanced usage
  • Typings
  • Scenarios
  • Circular Table dependencies

Motivation

When you have many dependent tables, it becomes difficult to create test data. Sometimes you need to test the function of changing a user's email address, but you need to create two more dependent tables because the user table has constraints.

In order to create user you have to create project and channel records first.

const { rows: project} = client.query({
  text: 'INSERT INTO projects(name, description) VALUES($1, $2)',
  values: ['Foo Project', 'Bla bla bla'],
})
const { rows: channel} = client.query({
  text: 'INSERT INTO channels(name, project_id) VALUES($1, $2)',
  values: ['dev-hunour', project.id],
})
const { rows: user} = client.query({
  text: 'INSERT INTO users(name, phone, foreign_id, channel_id) VALUES($1, $2)',
  values: ['brianc', '555-555-555', 123 channel.id],
})

rs-db-seeder allows you to do it in just one step

await dbSeeder.insert("user", { name: "john" });

Any missing information(phone, foreign_id fields) will be added and dependent records (channels, projects) will be automatically created behind the scenes.

Getting Started

DB Adapter

rs-db-seeder is framework agnostic. It doesn't use specific ORM - like knex, typeorm, sequelize. So you will need to build your own simple adapter for you application. The adapter implements IStorageWriter and has only one method insert. You can find example Postgres and Mysql adapter implementations here.

Here's a simple knex adapter for pg.

export class KnexPgStorageWriter implements IStorageWriter {
    private knex: Knex;
    constructor(knex: Knex) {
        this.knex = knex;
    }

    insert = async (tableName: string, data: any) => {
        const [result] = await this.knex(tableName).insert(data, "*");
        return {
            ...result,
            ...data,
        };
    };
}

Factories

Now that we have an adapter, we need to configure rs-db-seeder, so that it can generate test data and know about the dependent tables.

seeder.addFactory({
    id: "user", // unique factory ID
    tableName: "users", // table name
    dataProvider: (data: any): any => ({
        // data provider implementation
        id: 99,
        name: "John",
        phone: "55555555",
        foreign_id: 2132323,
        ...data,
    }),
    refs: [
        // identify dependent relations
        ref("channel"), // by default consider users.channel_id => {channel}.id
        // alternatively  you can do  `ref("channel", 'uuid', ch_id)` users.ch_id => {channel}.uuid
    ],
});
seeder.addFactory({
    id: "channel",
    tableName: "channels",
    dataProvider: (data: any): any => ({ name: "channel_1", ...data }),
});

Usage

// configure dbSeeder
const knex = configure();
const storage = new KnexPgStorageWriter(knex);
const dbSeeder = new DbSeeder(storage);

seeder.addFactory({
    id: "user",
    tableName: "users",
    dataProvider: (data: any): any => ({
        name: faker.name.firstName(),
        phone: faker.phone.phoneNumber(),
        ...data,
    }),
    refs: [ref("channel")],
});
seeder.addFactory({
    id: "channel",
    tableName: "channels",
    dataProvider: (data: any): any => ({
        name: faker.random.alpha({ count: 10 }),
        ...data,
    }),
});

// usage in your tests
import dbSeeder from "./configureDbSeeder";

it("updates user email", () => {
    const user = seeder.build("user", { name: "john" });

    const updatedUser = myUserMananger.updateName("tom");
    expect(updatedUser.name).toEqual("tom");
});

API

Build

Build operation allows you to build fake data for your entity. Data is not written to the database. It is somewhat like a faker, it just builds data for the entire entity. Note: data for referenced tables

const data = dbSeeder.build("user", { id: 100 });
// {
//   id: 100,
//   name: 'John',
//   phone: '55555555',
//   foreign_id: 2132323
// }

Note: data for referenced tables is not added. At the same time, no one bothers to add them ourselves.

const data = dbSeeder.build("user", {
    id: 100,
    channel: dbSeeder.build("channel"),
});
// {
//   id: 100,
//   name: 'John',
//   phone: '55555555',
//   foreign_id: 2132323
//   channel: {
//     name: "my channel"
//   }
// }

Insert

dbSeeder.insert - will build and write data to the DB. Note: it's async method. All referenced fields will be built and inserted as well, i.e. we will do 2 inserts into (ref) channels and users

const data = await dbSeeder.insert("user", { id: 100 });
// {
//    id: 100,
//    name: 'John',
//    phone: '55555555',
//    channel_id: 60,            // channel with ID = 60 has created
//    foreign_id: 2132323
//  }

Insert Many

dbSeeder.insertMany inserts multiple records

const [channel1, channel2] = await seeder.insertMany(2, "channel", {
    name: "channel_1",
});

inserts 2 channel records with name: channel_1

References

If you created a dependent entity before, pass it as a simple column value or as an object.

const channel = dbSeeder.build("channel");
const user1 = dbSeeder.build("user", { id: 100, channel_id: channel.id });
const user2 = dbSeeder.build("user", { id: 100, channel });

if you pass data as an object

const data = dbSeeder.build("user", {
    id: 100,
    channel: { name: "my channel" },
});

{ name: "my channel"} will be passed to channel insert method.

if you pass data as an object and an id field is specified, the dependent record will not be inserted.

const data = dbSeeder.build("user", { id: 100, channel: { id: 123 } });

Custom insert implementation

It's possible to provide custom insert implementation

seeder.addFactory({
    id: "user",
    tableName: "users",
    dataProvider: (data): any => ({
        id: 99,
        name: "John",
        phone: "55555555",
        foreign_id: 2132323,
        ...data,
    }),
    insert: async (data: any) => {
        const [user] = await knex("users")
            .insert(data, "*")
            .onConflict("foreign_id")
            .merge();
        return user;
    },
});

Clean up

The simplest and most efficient approach is to use uncommitted transactions. We insert data, do our tests, then inserted data is erased because the transaction is rolled back.

// use transaction instead if clean up
beforeEach(async () => {
    await knex.raw("BEGIN");
});
afterEach(async () => {
    await knex.raw("ROLLBACK");
});

it("insert data - simple case", async () => {
    const data = await seeder.insert("channel");
    ...
});

Why Seeder does not delete data automatically

Let's say you are testing the creation of an article. The article must have an author. (creator_id integer constraint article_users_id_fk references users)

afterEach(async () => {
    await dbSeeder.cleanup(); // this method doesn't not exist, see explanation below
});

const author = seeder.insert("author");
const article = postManager.addPost(...data, author);

Seeder knows that it inserted an author record. It doesn't know about article record. It has been added by postManager. If we try to delete data inserted by the Seeder we will delete only the author record and it leads to constraint violation since an article must have an author.

To avoid such conflicts, we abandoned the idea of automatic data deletion.

Advanced usage

Typings

type Factories = "users" | "channels";

const seeder: Seeder<Factories> = new DbSeeder(storage);
seeder.addFactory({
    id: "users",
    tableName: "users",
});
seeder.build("users");
seeder.insert("users");

// TS will signal errors
seeder.insert("usr");
seeder.build("usr");
seeder.addFactory({
    id: "user",
    tableName: "users",
});

Scenarios

For non-trivial cases, you can use scripts. Let's say you need to create records using transactions.

class AdvancedKnexPgStorageWriter extends KnexPgStorageWriter {
    getKnex() {
        return this.knex;
    }
}

const knex = getKnexPgClient();
const storage = new AdvancedKnexPgStorageWriter(knex);
const seeder: Seeder<"userWithChannel"> = new DbSeeder(storage);

seeder.addScenario({
    id: "userWithChannel",
    insert: async (
        storage: AdvancedKnexPgStorageWriter,
        data: any
    ): Promise<any> => {
        const knex = storage.getKnex();
        knex.raw("BEGIN");

        const channel = await storage.insert("channels", {
            name: data.channel.name,
        });

        const user = await storage.insert("users", {
            name: "John",
            phone: "55555555",
            foreign_id: randNumber(),
            channel_id: channel.id,
        });
        knex.raw("COMMIT");
        return { user, channel };
    },
});

Then you can use it in the same way as you use factories.

const { user, channel } = await seeder.insert("userWithChannel", {
    channel: { name: "Woh channel" },
});

Circular Table Dependencies

In case you have tables with circular dependencies

users(id, channel_id)    -> channel_id -> channels.od
channels(id, project_id) -> project_id -> projects.id
project(id, creator_id)  -> creator_id -> users.id

At least one table has nullable reference. Make sure you break the chain explicitly providing entity with nullable field.

const channel = await seeder.insert("channel", { project_id: null });
const user = await seeder.insert("user", { channel });