@status-machina/ddb-pattern

Warning: This package is unstable and experimental. Use it at your own risk. It has not been tested in meaningful production environments.

Overview

The @status-machina/ddb-pattern package provides utility functions and types for working with events and DynamoDB in a structured manner. It enables the creation, saving, and querying of events while leveraging ULID for unique identifiers and timestamping.

What is a "Stream" in this library?

A "stream" is a collection of events matching a specified type. Usually, a stream will also have unifying key matching some sort of "model".

Example: Todo List

As an example, to obtain a complete "stream" of events for a todo list, you would specify the event types as:

[
  TodoEventTypes.TODO_CREATED,
  TodoEventTypes.TODO_COMPLETED,
  // ...Any other todo-related events
  TodoEventTypes.TODO_DELETED
]

You would also specify the list_id as the model key.

Example: Todo Item

Alternatively, you could create a stream of events just for a single todo item, by specifying the event types as:

[
  TodoEventTypes.TODO_CREATED,
  TodoEventTypes.TODO_COMPLETED,
  // ...Any other todo-related events
  TodoEventTypes.TODO_DELETED
]

You would also specify the todo_id as the model key.

Example: User's archived todos

You could also create a stream of events for a user's archived todos, by specifying the event types as:

[
  TodoEventTypes.TODO_ARCHIVED,
]

You would also specify the user_id as the model key.

IMPORTANT NOTES

• This library intentionally does not expose methods for UPDATING or otherwise mutating events. It follows a write-only paradigm.
• This library is write-heavy, and it relies on duplicating events to improve lookup times. For systems with a constant high volume of events (1,000+ events/second), DDB is likely going to be more expensive than an equivalent system build on a relational DB.
• The responsibility is on the implementer to design their systems in a way that streams are relatively short (which is a key to being successful in an event-sourced system)

Purpose

This library is intended to create a better developer experience for projects that are event sourced and use DynamoDB as a storage medium. It borrows concepts from Single Table Design (primary key overloading, ULIDs for multipurpose/sortable IDs, etc).

It is heavily opinionated, in terms of structure.

Event Structure and Expectations

Events should adhere to a specific shape, encapsulated in the EventBase interface, which includes the following properties:

• id: A unique identifier for the event (A ULID generated by this library)
• type: The type of the event, which helps classify events in your system
• timestamp: The time when the event was created, formatted as an ISO string
• pk: The primary key used for partitioning the data in DynamoDB (Generated by this library)
• sk: The sort key used for ordering within the partition (A ULID, generated by this library)
• data: A record containing relevant event data, including any [model]_id key/value pairs which influence the lookup of the event.

It’s important to note that for every key in the data property that ends with _id, the event will be replicated and saved with a different partition key for each of those IDs. This ensures that events are organized and can be efficiently queried based on these identifiers.

Installation

You can install this package using npm:

npm install @status-machina/ddb-pattern

Usage

Define Your Event Types and Event Base

import { DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb";
import { EventBase, createEventFunctions } from "@status-machina/ddb-pattern";

// Define the DDB table name
const EVENTS_TABLE_NAME = process.env.EVENTS_TABLE_NAME;
if (!EVENTS_TABLE_NAME) {
  throw new Error(
    "Environment variable EVENTS_TABLE_NAME is required but not defined."
  );
}

// Define an enum of event types
export enum StoreEventTypes {
  PRICE_LOWERED = "PRICE_LOWERED",
  PRICE_RAISED = "PRICE_RAISED",
}

// Define a base event type to extend when defining your events
export type StoreEventBase = EventBase<StoreEventTypes>;

Use the Event Base to define the structure of your Events

// Define your own events
export interface PriceLowered extends StoreEventBase {
  type: StoreEventTypes.PRICE_LOWERED;
  data: {
    store_id: string;
    product_id: string;
    price: number;
  };
}

// Define your own events
export interface PriceRaised extends StoreEventBase {
  type: StoreEventTypes.PRICE_RAISED;
  data: {
    store_id: string;
    product_id: string;
    alert_id?: string;
    price: number;
  };
}

// Create a union of your events
export type StoreEvents = PriceLowered | PriceRaised;

Create an eventClient

// Create your event client
export const eventClient = createEventFunctions<StoreEventTypes, StoreEvents>(
  ddbClient,
  EVENTS_TABLE_NAME,
  "store_id",
);

// Example usage of the event client
async function example() {
  await eventClient.saveEvent({
    type: StoreEventTypes.PRICE_LOWERED,
    data: {
      price: 100,
      product_id: "01JA8B75A0NTKHTDPP7Q1W7Q35",
      store_id: "01JA8B7897ECZ47ZYM351HVBKB",
    },
  });

  const latestLoweredEventForStore = await eventClient.getLatestEvent({
    eventType: StoreEventTypes.PRICE_LOWERED,
    partitionId: "01JA8B7897ECZ47ZYM351HVBKB",
  });

  const latestLoweredEventForProduct = await eventClient.getLatestEvents({
    eventType: StoreEventTypes.PRICE_LOWERED,
    partitionId: "01JA8B7897ECZ47ZYM351HVBKB",
    modelKey: "product_id",
    modelId: "01JA8B75A0NTKHTDPP7Q1W7Q35",
    after: "01JA8B7K770W4XABTR20SSW99B",
  });

  const stream = await eventClient.getEventStream({
    eventTypes: [StoreEventTypes.PRICE_RAISED],
    partitionId: "01JA8B7897ECZ47ZYM351HVBKB",
  });
}

License

This package is licensed under the MIT License.