F🅻⌽𝕎↓⒠: connect functions as a flow

If you have a lot of functions and you want to connect them, monitor, and build complex flows without taking care about a lot of promises, flowie is the package for you.

Contents

• Getting started
• Creating modes
• Runtime API
  • Creating flows
  • Executing a flow
    • Initial Argument
    • Context Argument
    • The result
  • The flow API
    • .pipe
    • .split
    • FlowItem
    • Complex Example
• Prepared API
• Debugging
• Plans

Getting started

npm install flowie

const flowie = require('flowie')

async getJSONByUrl(url) { ... }

function getLatituteLongititudeOfISS({ iss_position: { latitude, longitude } }) {
  return { latitude, longitude }
}

async function whereIAm({ latitude, longitude }) {
    return getJSONByUrl(`https://mygeocode?q=${latitude},${longitude}`)
}

const flow = flowie(getJSONByUrl)
  .pipe(getLatituteLongititudeOfISS)
  .pipe(whereIAm)

const { lastResult } = await flow('http://api.open-notify.org/iss-now.json')
// the result of your geococode service

Running modes

There are two ways of creating a flow:

1. Runtime ➡️ flowie(...flowItemsList: FlowItem[]): Flowie: it create a flow with its own container.
2. Prepared ➡️ flowie(flowieContainer: FlowieContainer, preparedFlowie: PreparedFlowie): Flowie: it builds a flow from a declaration and uses this container.

a container is the object that flowie uses to store function details, i.e: name, isAsync, etc. When building using the prepared.

Both modes return Flowie flows, which can be execute or enhanced.

Runtime API

Creating flowie(...flowItemsList: FlowItem[]) : Flowie

This creates a flow that takes the initial argument then the "flow" starts.

Two or more functions means that it will works as a split FlowItem can be a function or another Flowie

Suppose you have these functions:

• isMyLuckyNumber: that receives number and returns boolean
• isABadLuckyNumber: that receives number and returns boolean
• isPositiveNegativeOrZero: that receives number and returns 'positive | negative | zero'

1. One function creation const flow = flowie(isMyLuckyNumber) // Flowie<number, boolean> flow will be a function that receives a number and return boolean

2. More than one function creation: const flow = flowie(isMyLuckyNumber, isABadLuckyNumber, isPositiveNegativeOrZero) // Flowie<number, [boolean, boolean, string*]> flow will be a function that receives a number and return an tuple [boolean, boolean, string*] we call this a split operation

3. Any time you can send function to flowie you can use antoher flowie, i.e

const flow1 = flowie(isMyLuckyNumber)
const flow2 = flowie(isABadLuckyNumber)
const flow3 = flowie(isPositiveNegativeOrZero)

const flow = flowie(flow1, flow2, flow3)

This result is the same example 2, but the execution is a bit different.

Executing a flow const flow: Flowie<Argument, Result, InitialArgument, Context> = flowie(...)(initialArgument)

flow(initialArgument: InitialArgument, context?: Context) Every flow receive can receive two arguments, the initialArgument, and context:

Initial Argument

This is provided as argument for the first FlowItems provided, does not matter if it is an function, more than one function (split), or a Flowie.

Context argument

If the some of the flow items need context object, you can receive as second argument, but all the flow items should use the same type for context, for instance:

function getUser(email: string, dbConnection: DbConnection): User {...}
function getMessages(fromEmail: string, dbConnection: DbConnection): Messages[] {...}
function getFilesOfUser(fromEmail: string, dbConnection: DbConnection): Messages[] {...}

const flow = flowie(getUser, getMessages)
/* getFilesOfUser would not be accepted,
 because the second argument(context),
 and it is not of the same for the previous flow items: getUser and getMessages */
const { lastResult } = flow('[email protected]', connectToMySql('mysql://127.0.0.1:3306'))
const [users, messages] = lastResult

The result of a flow const result = flow(initialArgument, context)

const flow: Flowie<Argument, Result, InitialArgument, Context> = flowie(...).pipe(...).split(...)
const result =  flow(initialArgument, context)

The result will have the following attributes:

• lastResult: this is the result of the last part of flow, pipe returns a single value, and split returns a tuple of all the result.
• executionTime: The total execution time of the flow in miliseconds
• functions: This is an object, where key is the name of a functions, and the values follow this structure:

const { functions } = flow(/* ... */)

/* functions.someFunction will something like */
{
  calls: 3, // total of calls
  slowestExecutionTime: 0.050902, // in miliseconds
  averageExecutionTime: 0.017708, // in miliseconds
  fastestExecutionTime: 0.000965, // in miliseconds
  totalExecutionTime: 0.053124, // in miliseconds, the sum of all executions
  iterations: { // just for generator functions
    count: 6, // total of iterations made
    slowestIterationTime: 0.135559, // in miliseconds
    averageIterationTime: 0.0388, // in miliseconds
    fastestIterationTime: 0.003701, // in miliseconds
    totalIterationTime: 0.232623 // in miliseconds, the sum of iterations
  }
}

The flow API Flowie<Argument, Result, InitialArgument, Context>

This is the result returned by flowie function, for instance: const flow: Flowie<string, boolean, number, never> = flowie(...)

It means, that this is a flow, that takes a number as first argument, and the last step of the flow is a flow item that receives a string and returns a boolean.

Flowie await method that are async

.pipe(flowItem: FlowItem)

As the name say it pipes content of previous step to the flowItem, let's say you have these objects:

  const myFlow1 = flowie()
  const myFlow2 = flowie()
  const aFunction = () => {}
  const otherFunction = () => {}

You can play with them they way you want: flowie(aFunction).pipe(myFlow1).pipe(myFlow2).pipe(otherFunction). This would generate a flow with four steps, executing them in order.

.split(...flowItemList: FlowItem[])

Splitting on flowie is a step that receives one argument call more them on flowItem in paralell,

if you create a flowie with more then one function, it means start splitting, as mentioned Here

The step after a splitting will receive the tuple of result of the split:

const add50 = (x:number) => x + 50
const add10 = (x:number) => ({ plus10: x + 10 })
const add20 = (x:number) => ({ plus20: x + 20 })

const whatDoIReceive = (question: any) => console.log(question)

const flow = flowie(add50).split(add10, add20).pipe(whatDoIReceive)
flow(10)
// this would log [{ plus10: 70 }, { plus20: 80 }]
// I hope the math was right 😬

FlowItem

A flow item is the node of a flow, it can be a function, an async function, a generator function, an async generator function or a Flowie Flow. Unfornately FlowItem don't work well with spread(...) operator

1. Functions

When piping or splitting, you can use functions, the result type will be the argument of the next flowItem on the flow.

2. Async functions

When piping or splitting, you can use async functions, the thenArgType returned by function will be the argument of the next flowItem on the flow.

By now... just functions using async declarations are accepted as flowItem, so... functions like this one function doAsyncOperation() { return new Promise<type>(...) } will be recognized as a function.

If one of the functions of the flow is async, than result becomes a promise.

3. Generator functions, and async generator functions

Generator functions, async or not... like function* generator() {} or asyncfunction* generator() {} are considered as generator functions on flowie. Every flowItem piped after the same a flow item that is a generator function will be called once per yield, and the result will be the last value.

For instance:

function* generatorLetters(count: number) { ... } // yields from A to Z, limited by count. Count: 3, yields A,B, and C
async function* getUserStartingWith(letter: string) { ... } // yields users that start with the letter
async function sendEmailMarketing(user: User) { ... } // sends a mail marketing to a user

const sendEmailMarketingFlow = flowie(generatorLetters).pipe(getUserStartingWith).pipe(sendEmailMarketing)
sendEmailMarketingFlow(5).then(() => console.log('Emails sent to users starting with A-E!'))
// the generator function getUserStartingWith will be invoked 5 times
// sendEmailMarketing will be invoked for every user starting with letters A,B,C,D and E

If one of the functions of the flow is async generator, than result becomes a promise.

By splitting with a generator, like flowie(generator, nonGenerator) or flowie.split(generator, nonGenerator), this will not affect next steps, so:

const flow = flowie(generator,nonGenerator).pipe(howManyTimesAmIGoingToCalled)
flow()

Will cause the function howManyTimesAmIGoingToCalled to be called just once, but the iterator returned by generator will be consume completely.

4. Flowie as flowItem

This is done to make possible re-use other flows and also build complex flows, see example below:

const sendEmailMarketingFlow = flowie(prepareEmailMarketing).pipe(sendEmailMarketing)
const sendSMSMarketingFlow = flowie(prepareSMSMarketing).pipe(sendSMSMarketing)

async function* getUsers(dbConnection) {...}

const sendAllMarketingFlow = flowie(getUsers).split(sendEmailMarketingFlow, sendSMSMarketingFlow)
await sendAllMarketingFlow(dbConnection)

The flow above: sendAllMarketingFlow, would prepare and send emails and SMS for every user yield by getUsers.

Other examples

const detectsABC = (equation: string) => [a,b,c] // all number
const calculateDelta = ([a,b,c]: [number, number, number]) => [a,b,c, delta]
const returnsX1Result = ([a,b, delta]: [number, number, number]) => { x1: number, delta }
const returnsX2Result = ([a,b, delta]: [number, number, number]) => { x2: number, delta }
const mergeObjects (objects: any[]) => Object.assign({}, ...objects)

const secondDeegreeResolverFlow = flowie(detectsABC)
  .pipe(calculateDelta)
  .split(returnsX1Result, returnsX2Result)
  .pipe(mergeObjects)

const { lastResult } = secondDeegreeResolverFlow('1x² - 3x - 10 = 0')
console.log(lastResult) // {delta: 49, x": -2, x': 5}

See the execution on runkit, clicking here

Prepared API

Debugging

The only dependency that flowie have is debug, in order to help you to see what is happening. The namespace is flowie, so DEBUG=flowie*, as the value of the variable DEBUG will active the debug.

One extra feature is, if the namespace debugFlowie is enabled, than a debugger statement will be included on the first line of the flow that will be executed.

Plans

There is no priorization on this list yet

• [x] Pipe/Split (Runtime and Prepared)
• [x] Async Pipe/Split (Runtime and Prepared)
• [x] Accept flowie as flowItem
• [x] Check on runkit
• [x] Accept generators on pipe/split
• [x] Accept async generators on pipe/split
• [x] Context Parameter
• [x] add Debug library calls and debugger statement to flows
• [x] Reporting (timePerFunction, numberOfCalls, slowestExecution, AvgExecution, fastestExecution)
• [ ] Event Emitter
• [ ] Validate prepared flowie, and function names on container
• [ ] When there is two functions with same name, add a suffix
• [ ] add Flags (actAsGenerator, actAsAsync) on .pipe/.split in order to be able to receive functions that returns () => Promise.resolve() or iterators () => { [Symbol.iterator]: () => {} }
• [ ] Validate flow declaration on prepared mode
• [ ] Detect recursion flowie on runtime
• [ ] Validate parameteres on FlowItems
• [ ] Global Error Handling (interrupt flow or not)
• [ ] Error Handling for split/generators
• [ ] Backpressure for generator
• [ ] Batching***
• [ ] Limit concurrency on split
• [ ] Enhance reports (custom prepared, log input/output)***
• [ ] Filter flowItem (FlowItem that 'stop' current flow or subFlow)
• [ ] Report flowItem (bypass argument, but is called)
• [ ] Decider flowItem (like split, with names, based on argument, decide which flow ot execute)
• [ ] ChangeContext flowItem (changes the context on item below the same flow)
• [ ] Mechanism to verify inputs/outputs