Twired

• Separate business logic and runtime concerns.
• Tiny (< 5kb) library with 0 dependencies.
• Write distributed code that look and feel like regular code.
• Easy to migrate — just write an executor adapting your current job/queue/task system. Doable in an afternoon!
• Swap out executors in different environments for different behaviors and guarantees.

Problem Statement

When a codebase evolves to make use of distributed computing, they often have to change how code is written and structured. What used to be a simple function call is turned into createTask, runTask, collectTask calls, often split up into different directories and utilities.

This makes your codebase hard to understand, slow to debug, and difficult to develop.

You probably even adopt different systems or services for different characteristics and guarantees, multiplying the surface area for developers to understand — hundreds of createTask, sendMessage, startJob, each probably using the same (hopefully?) underlying functions nested deep inside wrapped calls and workers.

This library is written as an answer to this problem.

Experience again writing simple function calls with semantic clarity and type safety.

Port your existing job systems, message queues, run times to an executor you can write in an afternoon. Re-use all of your existing infrastructure, and test them side-by-side!

Run the same functions with different guarantees and characteristics depending on the environment and your needs!

Usage

1. npm install twired
2. move your function into a Twired class / extend your class with an executor property
3. decorate your function with @dispatch or @dispatchAwait
4. use / write an executor to run the decorated functions
5. until decorators are accepted into Ecmascript, you will need a transpiler, e.g. babel / typescript

Examples

Each of the directories in examples are executable.

Check out each of their readmes!

1. Local Executor (this is a minimal example wrapping a regular local function call)
2. Redis Executor (this is an example which support different dispatch and dispatchAwait calls, and separating calling and running functions)
3. MQ Executor (an example showcasing only dispatch type calls, validating executor configuration, and resuming of jobs)

import { Twired, dispatch, dispatchAwait } from "twired";

class Greeter extends Twired {
  @dispatch
  async greet(recipient: string) {
    const person = await getPerson(recipient);
    const message = await this.generateGreeting(person.name);
    await this.sendGreeting(recipient, message);
  }

  @dispatchAwait
  async generateGreeting(name: string) {
    return `Hello ${name}`;
  }

  @dispatch
  async sendGreeting(recipient: string, message: string) {
    const messageId = await sendEmail(recipient, message);
    await this.saveMessage(messageId);
  }

  @dispatch
  async saveMessage(messageId: string) {
    await saveEvent(messageId);
  }
}

async function main() {
  const executor = new RedisExecutor();
  const greeter = new Greeter(executor);
  const greeting = await greeter.greet("[email protected]");
}

Concepts

Inspired by this paper

dispatch and dispatchAwait decorators

These decorators wrap functions and allow executors to execute them.

dispatch is meant for function calls where you do not expect a response, e.g. saving events without waiting for its response. The type signature enforces a void return value.

dispatchAwait is meant for function calls where you expect a response, e.g. an API server recevies a request, wants to offload processing to a different machine / thread, and receive a result to create a response.

You will notice the distinction is arbitrary. You can technically use dispatchAwait for all calls, even for those without a response. I find it useful to make this explicit to understand the decorated function's behavior at a glance.

Executors can also make use of this distinction to optimize wrapped calls, e.g. an executor using messages can make dispatch calls return when a message is acknowledged without waiting for the result; an "unsafe" executor can even immediately return dispatch calls without waiting for the queue to acknowledge the message.

Why decorators?

Decorators for this use case strike a good balance of transprency (it's obvious that this method is not a regular function call) and lack-of-clutter. You can easily follow function calls without having to jump through higher order functions or different runners/workers.

Executors

The Executor interface defines how an executor deals with dispatch and dispatchAwait decorated functions.

Executors are expected to implement at least the call method, which represents how it deals with a function call from the decorators.

Executors can implement register, which will be called on decorator initialization. At the moment, this is the method used by executors to do anything outside of the function call, such as validate options per function call (e.g. needing a specified queue per decorated function), subscribing to whatever run time job/message system, processing work, routing, etc.

Advanced executors could have fallbacks to local execution, or on-the-fly adjustment of where to execute code depending on resource usage or desired durability requirements.

Twired

The Twired base class simply ensures that there is an executor attached to the decorated class methods, and that register initialization works out of the box.

You can also implement the interface HasExecutor, or even avoid all of it if you can ensure you have the executor property on your classes on decorator initialization. This requires knowing a bit of decorator-specific knowledge (ref).

TODOs

• [ ] Error types / cases
• [ ] Thread/worker example
• [x] Redis example
• [x] MQ example
• [x] Executor with options example
  • [x] MQ executor enforces each decorated method needs a distinct queue name