parallelize-generator-promises
v1.1.5
Published
Runs promises yielded by generators in parallel
Downloads
23
Readme
parallelize-generator-promises
A simple utility to run promises yielded by a generator in parallel. This is incredibly useful when you need to run a bunch of tasks as fast as possible without waiting for one another.
Installation
parallelize-generator-promises is available on both npm and JSR.
To use from npm, install the parallelize-generator-promises package and then import into a module:
import { parallelizeGeneratorPromises } from "parallelize-generator-promises";
To use from JSR, install the @reda/parallelize-generator-promises package and then import into a module:
import { parallelizeGeneratorPromises } from "@reda/parallelize-generator-promises";
Polyfilling Promise.withResolvers
If you are using Node.js v20 or below, you will require a polyfill for
Promise.withResolvers
.
A polyfill for this is available in a separate export as part of this package. To use the provided polyfill, import it at the entrypoint of your application.
npm:
import "parallelize-generator-promises/promise-with-resolvers-polyfill";
jsr:
import "@reda/parallelize-generator-promises/promise-with-resolvers-polyfill";
Usage
import { parallelizeGeneratorPromises } from "parallelize-generator-promises";
async function* getAllProductDetails() {
for (let page = 1; page <= 100; page++) {
const products = await fetch(`/products?page=${page}`).then((res) =>
res.json()
);
yield products.map((product) =>
fetch(`/product/${product.id}/details`).then((res) => res.json())
);
}
}
for await (
const productDetails of parallelizeGeneratorPromises(
getAllProductDetails(),
)
) {
console.log(productDetails);
}
The above example will fetch all product details for one page of products at a time as fast as possible, yielding the product details in the fastest order possible to keep the iterator fed.
If a concurrency limit is required, this utility pairs very well with a semaphore library such as async-sema.
Options
| Name | Description |
| :-------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| maxBufferedPromises
| Limits the maximum number of promises that can be buffered at any given time. Useful to manage memory usage in the case where you are generating a lot of promises that aren't being consumed at a fast enough rate. NOTE: this value must be greater than or equal to 1. By default this is undefined
which means there is no limit set. |