throttlewrap

A convenient wrapper for adaptively throttling async functions

Installation:

npm i throttlewrap

Usage

tw.wrap([functionToThrottle], options)

It returns a wrapped version of functionToThrottle with rate limitation described in the options object.

Simple uses

rpm limitation on a callback funtion:

const tw = require('throttlewrap');
const request = require('request');

const get = tw.wrap(request.get, { rpm: 10 });

get('http://myurl.com', console.log);
get('http://myurl.com', console.log);
get('http://myurl.com', console.log);
get('http://myurl.com', console.log);

simultaneous thread limitation on a promise function:

const tw = require('throttlewrap');
const request = require('request-promise');

const get = tw.wrap(request.get, { threads: 2 });

get('http://myurl.com').then(console.log, console.error);
get('http://myurl.com').then(console.log, console.error);
get('http://myurl.com').then(console.log, console.error);
get('http://myurl.com').then(console.log, console.error);

Using rules to adaptively throttle

using an object rule to speed up when no errors for half a second

const get = tw.wrap(request.get, {
  rpm: 10,
  rules: [{
    condition: { noErrorPeriod: 500 },
    action: { rpm: { mul: 2 } },
  }]
});

using an object rule to remove threads when more than 20% of the responses are errors or timeouts in a 2 second period

const get = tw.wrap(request.get, {
  threads: 5,
  isError: (err, res) => err || res.statusCode === 504,
  rules: [{
    condition: { errorRate: { gt: 0.2, period: 2000 } },
    action: { threads: { sub: 1 } },
  }]
});

A complex example

const get = tw.wrap(request.get, {
  threads: 10,
  threadsMin: 2,
  threadsMax: 20,
  rps: 15,
  rpmMin: 300,
  intervalMax: 500,
  isError: (err, res) => err || res.statusCode === 504,
  rules: [{
    condition: {
      noSuccessPeriod: 500
    },
    action: {
      threads: { sub: 1 },
      rpm: { div: 1.5 }
    },
  },
  {
    condition: {
      successCount: { gte: 5, period: 500 }
    },
    action: {
      threads: { add: 1 }
    },
  },
  function ({ lastErrorTime, firstCallTime, interval }) {
    const now = Date.now();
    const noErrorPeriod = now - (lastErrorTime || firstCallTime);
    const shouldSpeedUp = noErrorPeriod >= 2000
      && (!this.lastApplied || (now - this.lastApplied > 2000));
    if (!shouldSpeedUp) return null;
    this.lastApplied = now;
    return { interval: interval / 1.5 };
  }]
});

Table of options

None of these options are required, but you need to define the function to throttle and one of rpm, rps, interval or threads

|key|type|description| |---|---|---| |interval|number|The number of milliseconds that need to pass between each call |rpm|number|The maximum of calls per minute. Gets converted to interval |rps|number|The maximum of calls per second. Gets converted to interval |threads|number|The maximum number of simultaneous calls. When omitted or set to 0, the number of simultaneous calls are unlimited |type|string|Values can be 'promise' or 'callback', describing the type of function wrapped. If omitted, the type will be determined for each call by checking if the last argument passed is a function (callback) |fn|async function|The function to be wrapped. Can also be defined as the first argument of the throttlewrap.wrap function. |rules|array|Describes the conditions on which execution should speed up / slow down. Increasing and decreasing the number of threads is also possible. See the rules section describing these in detail |isError|function|Defines what results to treat as errored. Runs after each finished call, receives the error and result of the call. The 3rd parameter will be true if a promise is rejected, this is useful when it gets rejected with no error passed. If isError returns truthy value, the response will be treated as an error in the stats. Default value is (err, res, rejected) => err || rejected |intervalMin|number|The smallest number interval can be adjusted to using rules |intervalMax|number|The largest number interval can be adjusted to using rules |rpmMin|number|The smallest number rpm can be adjusted to using rules. Will get converted to intervalMax |rpmMax|number|The largest number rpm can be adjusted to using rules. Will get converted to intervalMin |rpsMin|number|The smallest number rps can be adjusted to using rules. Will get converted to intervalMax |rpsMax|number|The largest number rps can be adjusted to using rules. Will get converted to intervalMin |threadsMin|number|The minimum number threads can be adjusted to using rules. Defaults to 1 where a function is wrapped using threads limitation. |threadsMax|number|The maximum number threads can be adjusted to using rules. |statsPeriod|number|The number of milliseconds for which to keep the statistics for. If omitted, it will be determined by checking the conditions in the rules array

The rules array

An array of objects and/or functions describing when and how to adapt throttling. While function rules give more flexibility, in most cases using object rules is more convenient.

Object rules

Each rule object needs to have two keys, condition and action. These will get processed after each finished call, where the condition is met, the action will be taken.

Condition object

Defines the condition on which the action is to be taken. Needs to have at least one of the below keys. If more keys are defined there will be an AND relation between the multiple conditions. In order to use an OR relation, please define them as separate rules.

|key|type|example|description| |---|---|---|---| |noErrorPeriod|number|{ noErrorPeriod: 500 }|Condition will be met if no call will finish with error for the number of milliseconds. This is half a second in the example. |noSuccessPeriod|number|{ noSuccessPeriod: 2000 }|Condition will be met if all calls finish with error for the number of milliseconds. This is two seconds in the example. |errorRate|object|{ errorRate: { period: 1000, gt: 0.5 } }|Will trigger the action if the ratio of errors satisfy the condition in the period. In this example the condition will be met if more than half of the finished calls in the last second errored. period is required, operator key can be gt - greater than, gte - greater or equal, lt - less than, lte - less or equal, is - equals. Values will fall between 0 and 1 inclusive. |successRate|object|{ successRate: { period: 800, lte: 0.25 } }|Same as the above but checks the ratio of successful (not errored) calls. In this example the condition will be met if in the past 800 milliseconds only 25% or less of the calls finished without error. period is required, operator key can be gt, gte, lt, lte, or is |errorCount|object|{ errorCount: { period: 500, is: 3 } }|Similar to errorRate but rather than the ratio, it looks at the number of errors. In this example the condition will be met if in the past half a second 3 calls finished with errors. period is required, operator key can be gt, gte, lt, lte, or is |successCount|object|{ successCount: { period: 1000, gte: 5 } }|Same as the above but checks the count of successful (not errored) calls. In this example the condition will be met if in the past second the number of successful calls is 5 or more. period is required, operator key can be gt, gte, lt, lte, or is

Action object

Defines what to do when the condition is met. Needs to have at least one of the below keys. If more keys are defined all of the multiple actions will be processed. Key values are objects in the format of { operator: value }

|key|example|description| |---|---|---| |threads|{ threads: { sub: 1 } }|Changes the number of threads. (The maximum number of simultaneous calls) This example action will remove one thread. Operators can be mul - multiply, div - divide, add - increase, sub - subtract, set - set to exact value |interval|{ interval: { set: 100 } }|Changes the interval, the number of milliseconds between calls. This example action will set the interval to 100, allowing a maximum of 10 calls per second. Operators can be mul, div, add, sub or set |rpm|{ rpm: { mul: 2 } }|Changes the rate per minute limit. In this example action the rpm gets doubled. Operators can be mul, div or set |rps|{ rps: { set: 5 } }|Similar to the above, but adjusts the rate per second. In this example action the rps gets set to 5. Operators can be mul, div or set

Function rules

A rule can also be defined as a function describing the condition and action when using an object rule is not flexible enough. Each function rule will be executed after each finished call. The rule function will get an object as an input parameter, object keys are described in the table below. If the rule is to change the throttling parameters, it has to return an object with the to be modified key(s) and its new value. Note that rpm and rps are not in the object the rule function receives, and will not be processed if returned. Please use interval instead.

|key|type|description| |---|---|---| |q|array|The pending calls waiting for execution. Each call is an object describing the arguments the function needs to be called with, the type of call (callback|promise) and a callback that will be called once the function finished |errorTimes|array|An array of timestamps when the throttled function errored. It will only go back as far as it is defined in the statsPeriod |successTimes|array|An array of timestamps when the throttled function responded without error. It will only go back as far as it is defined in the statsPeriod |fn|async function|The function being throttled |threads|number|The number of threads - maximum number of simultaneous calls. Will be undefined if unlimited simultaneous calls are allowed |interval|number|The number of milliseconds that need to pass between each call |intervalMin|number|The smallest number interval can be adjusted to using rules |intervalMax|number|The largest number interval can be adjusted to using rules |rpmMin|number|The smallest number rpm can be adjusted to using rules. This cannot be returned by the function rule, please use intervalMax instead |rpmMax|number|The largest number rpm can be adjusted to using rules. This cannot be returned by the function rule, please use intervalMin instead |rpsMin|number|The smallest number rps can be adjusted to using rules. This cannot be returned by the function rule, please use intervalMax instead |rpsMax|number|The largest number rps can be adjusted to using rules. This cannot be returned by the function rule, please use intervalMin instead |threadsMin|number|The minimum number threads can be adjusted to using rules. |threadsMax|number|The maximum number threads can be adjusted to using rules. |isError|function|The function that checks if a call's response is to be treated as an error in the stats |type|string|Values can be 'promise' or 'callback', describing the type of function wrapped. |statsPeriod|number|The number of milliseconds for which to keep the statistics for. |lastErrorTime|number|The timestamp when the wrapped function last finished with error |lastError|any|The error returned from the last call. Will not have a value if the last call responded without error |lastSuccessTime|number|The timestamp when the wrapped function last finished without error |lastResult|any|The result returned from the last call. Will not have a value if the last call errored |firstCallTime|number|The timestamp when the wrapped function was first executed |nextFreeTimeslot|number|A timestamp for the next possible call to be scheduled to. This is not to be modified by the rule function, use interval/rpm/rps instead |lastCall|object|The arguments, type and callback of the last wrapped function call. This is written when the wrapped function is called, not when the actual execution started. It is the last element of the q array.