forEachQueue

A tool for asynchronously working through large lists while keeping things a little more controlled than await Promise.all().

Example

A hypothetical example showing a lookup of user ratings for 50,000 movies. The limit is set to 50 pending requests at once so we don't get rate limited or run into timeouts.

import forEachQueue from "@ratwizard/for-each-queue";

const lotsOfItems = [
  /* 50,000 item array */
];

const movieScores = await forEachQueue(lotsOfItems, 50, async (item) => {
  // Make web requests, for example. A maximum of 50 requests will be pending at once in this case.
  // When one finishes the next will be queued up.
  const results = await http.get(
    `https://movie-api.com/api/stuff?t=${item.name}`
  );
  return {
    name: item.name,
    score: results.userRating,
  };
});

for (const movie of movieScores) {
  console.log(`${movie.name}: ${movie.score}`);
}