npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@jupiterone/hierarchical-token-bucket

v0.3.1

Published

A simple hierarchical token bucket

Downloads

21,707

Readme

@jupiterone/hierarchical-token-bucket

This project exports a HierarchicalTokenBucket class that can support nested rate limits. This should be used in client-side rate limiting strategies in order to honor rate limits that are composed in a nested structure. One such example is AWS API rate limits, which can be limited by an account-level, service-level, region-level, or API-level bucket.

The token bucket returns a numeric timeToWaitInMs from its primary interface, .take(). This allows the token bucket to remain synchronous, so it does not block other requests. Each caller is expected to honor the timeToWaitInMs returned from .take().

Returning a timeToWaitInMs when the bucket is already exhausted, rather than simply preventing the caller from take()ing a token and forcing it to re-call, essentially creates a lightweight FIFO queue where each caller invokes the interface just one time.

Usage:

import { HierarchicalTokenBucket } from '@jupiterone/hierarchical-token-bucket';

async function sleep(ms: number) {
  return new Promise(r => setTimeout(r, ms));
}

const parentBucket = new HierarchicalTokenBucket({
  maximumCapacity: 100,
  refillRate: 10
});

const childBucket = parentBucket.child({
  maximumCapacity: 10,
  refillRate: 1,
});

const timeToWaitInMs = childBucket.take();

await sleep(timeToWaitInMs);
await fetch('https://my.rate-limited.resource');

Alternately, this can be simplified by invoking withTokenBucket.

import { 
  HierarchicalTokenBucket,
  withTokenBucket
} from '@jupiterone/hierarchical-token-bucket';

const tokenBucket = new HierarchicalTokenBucket({
  maximumCapacity: 100,
  refillRate: 10
});

const cb = () => fetch('https://my.rate-limited.resource');
await withTokenBucket(tokenBucket, cb);

One can also specify a child without passing options, in which case maximumCapacity and refillRate are inherited from the parent bucket. This means that the child bucket will not limit usage any more than the parent bucket would, but it might be useful when instrumenting code for optional limiting.

Class: HierarchicalTokenBucket

new HierarchicalTokenBucket(params)

  • params.maximumCapacity {number} The total number of requests allowed when the bucket is full.
  • params.refillRate {number} The number of requests to add to the bucket per second. The bucket will never exceed maximumCapacity requests.

tokenBucket.take()

Takes a token from this and all parent token buckets. Returns the number of milliseconds that must elapse before attempting to redeem the token. Returns 0 if the token can be redeemed immediately.

Consumers need only call this function once, but may need to wait before redeeming their token.

const timeToWaitInMs = hierarchicalTokenBucket.take();

if (timeToWaitInMs > 0) {
  await new Promise(r => setTimeout(r, timeToWaitInMs));
}

await fetch('https://my.target.host/that/supports/throttling')

tokenBucket.metadata

Returns the token bucket metadata, including

  • options.maximumCapacity
  • options.refillRate
  • metrics.firstTakeTimestamp
  • metrics.takeCount

This metadata can be used to adjust the token bucket options in the event that a rate-limited request is encountered. For example:

try {
  const timeToWaitInMs = tokenBucket.take();
  await sleep(timeToWaitInMs);
  await client.request();
} catch (err) {
  if (isRateLimitError(err)) {
    const { options, metrics } = tokenBucket.metadata;
    logger.warn({
      maximumCapacity: options.maximumCapacity,
      refillRate: options.refillRate,
      firstTakeTimestamp: options.firstTakeTimestamp,
      takeCount: metrics.takeCount,
    }, 'Encountered rate limited request. Operator should adjust token bucket maximumCapacity or refillRate.');
  }
}