promise-pool-ext

Queue promises into pool limiting concurrency

Install

Install with npm:

$ npm install --save promise-pool-ext

Usage

Pool

import { Pool } from 'promise-pool-ext';

const pool = Pool({ concurrency: 10 });

// queue array of functions, returns Promise < array of function results >
pool([
  () => new Promise((resolve) => { /* do async logic here */ }),
  async () => { /* do async logic here */ }
]);

// queue function, returns Promise < function result >
pool(async () => { /* do async logic here */ });

See tests for more examples

PoolManager

import { PoolManager } from 'promise-pool-ext';

const manager = PoolManager({
  check1: {
    if: () => false, // when not true returned, promise is not resolved and undefined is returned
    fn: () => { /* return async logic */ }
  },
  check2: {
    requires: ['check1'],
    fn: ({ check1 }) => {
      if (check1) { /* return async logic */ }
    }
  },
  data: {
    requires: ['check1', 'check2'],
    fn: ({ check1, check2 }) => {
      if (check1 && check2) { /* return async logic */ }
    }
  }
}, { concurrency: 10 });

// returns Promise < data.fn result >
manager.get('data');

The requires option can be set to "*" to make all previous tasks required.

See tests for more examples

SeqWorker

Allows sequential handling of async tasks that are enqueued.

enqueue()

Used to enqueue an (async) task. This task will be executed once all previous tasks have executed.

flush()

Can use flush to finish all pending tasks in order. The next task is only executed once the previous one has completed.

See tests for more examples.

Parameters

concurrency: Integer

How many promises can resolve in parallel.

Not supported for SeqWorker (since it is always one)

timeout: Integer = undefined

Maximum amount of time in ms that a promise can take to resolve before a failure is returned.

If 0 or undefined is passes, no timeout is enforced.

debounce: Boolean = undefined

Only supported for SeqWorker.

When set to true, will skip any queued tasks if there are subsequent ones.

Errors

When a promise is rejected or an error is thrown, the returned promise is rejected once all promises in batch have completed execution.

Why

As an example, when making a lot of external requests one can hit limits of the external API or the node runtime itself. This library makes it easy to limit the amount of parallel executions.