async-delay-queue

Minimal ES7 async queue with dynamic delay between functions.

Usage

queue.delay(fn, delay, timer, add)

Returns a promise resolving the return value of the given function.

| Argument | Description | Default | |:------------- |:------------- |:------------- | | fn | Function to delay. Supports promise/async functions. | None | | delay | Delay between this function and the last one in ms. Required. | null | | timer | Max time until the active promise is forcefully resolved to prevent the queue getting stuck. | null | | add | Method with which the function is added to the queue stack. "unshift" to add to start, "push" to add to end of queue. | "push" |

Example

Crawl Google.com and retry when hitting rate limits.

const queue = require("async-delay-queue")
const request = require("request-promise") // for example purposes only

async crawl(url) {
  let res = await queue.delay(() => request(url), 100)

  // Hit rate limits? Put the same request at the start of the queue.
  // "unshift" to push at start, and increase delay to 10s.
  if (res.statusCode === 429) {
    res = await queue.delay(() => request(url), 10000, null, "unshift")
  }

  return res
}

// Crawls google.com 20 times with a 100ms delay between each request.
for (let i = 0; i < 20; i++) {
  crawl("http://google.com")
}