npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

puzzle-warden

v1.8.6

Published

<p align="center"><img width="400" alt="Warden" src="./logo.png"></p>

Downloads

2,722

Readme

Warden is an outgoing request optimizer for creating fast and scalable applications.

CircleCI npm npm Known Vulnerabilities codecov Codacy

Features

  • 📥 Smart Caching Caches requests by converting HTTP requests to smart key strings. ✅
  • 🚧 Request Holder Stopping same request to be sent multiple times. ✅
  • 🔌 Support Warden can be used with anything but it supports request out of the box. ✅
  • 😎 Easy Implementation Warden can be easily implemented with a few lines of codes. ✅
  • 🔁 Request Retry Requests will automatically be re-attempted on recoverable errors. ✅
  • 📇 Schema Stringifier Warden uses a schema which can be provided by you for parsing JSON stringify. ✅
  • 📛 Circuit Breaker Immediately refuses new requests to provide time for the API to become healthy. ✅
  • 🚥 API Queue Throttles API calls to protect target service. 📝
  • 👻 Request Shadowing Copies a fraction of traffic to a new deployment for observation. 📝
  • 🚉 Reverse Proxy It can be deployable as an external application which can serve as a reverse proxy. 📝

Warden Achitecture

Getting started

Installing

Yarn

yarn add puzzle-warden

Npm

npm i puzzle-warden --save

Quick Guide

1. Register Route

const warden = require('puzzle-warden');
warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

2. Send Request

Using Route Registration
const routeRegistration = warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

routeRegistration({
  url: `https://postman-echo.com/get?foo=value`,
  headers: {
    cookie: `bar=value`
  },
  method: 'get',
  gzip: true,
  json: true
}, (err, response, data) => {
  console.log(data);
});
Using Warden
warden.request('test', {
  url: `https://postman-echo.com/get?foo=value`,
  headers: {
    cookie: `bar=value`
  },
  method: 'get',
  gzip: true,
  json: true
}, (err, response, data) => {
  console.log(data);
});
Using Request Module
request({
  name: 'test',
  url: `https://postman-echo.com/get?foo=value`,
  headers: {
    cookie: `bar=value`
  },
  method: 'get',
  gzip: true,
  json: true
}, (err, response, data) => {
  console.log(data);
});

Identifier

Warden uses identifiers to convert HTTP requests to unique keys. Using these keys it is able to implement cache, holder and other stuff. Let's assume we want to send a GET request to https://postman-echo.com/get?foo=value&bar=anothervalue. And we want to cache responses based on query string foo. We should use the identifier {query.foo}. There are 5 types of identifier variables.

  • {url} Url of the request
  • {cookie} Cookie variable. You can use {cookie.foo} to make request unique by foo cookie value.
  • {headers} Header variable. You can use {headers.Authorization} to make request unique by Authorization header
  • {query} Query string variables. You can use {query.foo} to make request unique by query name.
  • {method} HTTP method. GET, POST, etc.

You can also use javascript to create custom identifiers.

  • {url.split('product-')[1]} Works for link /item/product-23.

Identifiers can be chained like {query.foo}_{cookie.bar}.

Identifiers get converted to keys for each request. Let's assume we have an identifier like {query.foo}_{method} We use this identifier for a GET request to /path?foo=bar. Then the unique key of this request will be bar_GET.

Registering Route

You can simply register a route providing an identifier and module configurations. Please see Identifier

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

identifier is an optional field. If an identifier is not provided warden will be use generic identifier which is ${name}_${url}_${JSON.stringify({cookie, headers, query})}_${method}.

Cache

You can simply enable cache with default values using.

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

Or you can customize cache configuration by passing an object.

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: {
    plugin: 'memory',
    strategy: 'CacheThenNetwork',
    duration: '2m'
  },
  holder: true
});

Default values and properties

| Property | Required | Default Value | Definition | | :--- | :---: | ---: | :--- | | plugin | ❌ | memory | Where cached data will be stored. Please see Cache Plugins for more information. Currently, only memory available. | | strategy | ❌ | CacheThenNetwork | Controls when and how things will be cached. Please see Caching Strategy for more information. | | duration | ❌ | 1m | Caching Duration. You can use number for ms. Or you can use 1m 1h 1d etc. Please see ms for full list| | cacheWithCookie | ❌ | false | Warden never caches responses including set-cookie header. To enable this pass this property as true |

Cache Plugins

Cache plugins control where cache will be stored. These are available plugins:

Custom Plugins

Anything that implements interface below can be used as Cache Plugin.

interface CachePlugin {
  get<T>(key: string): Promise<T | null>;

  set(key: string, value: unknown, ms?: number): Promise<void>;
}

You first register the cache plugin

warden.registerCachePlugin('mySuperCache', {
  set(){},
  get(){}
});

Then make route configuration with your plugin name

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: {
    plugin: 'mySuperCache',
    strategy: 'CacheThenNetwork',
    duration: '2m'
  },
  holder: true
});

Caching Strategy

Caching strategies defines how things will be cached and when cached responses will be used. Currently, the only available caching strategy is CacheThenNetwork

CacheThenNetwork

Simple old school caching. Asks cache plugin if it has a valid cached response. If yes, returns the cached value as the response. If no, passes the request to the next handler. When it receives the response, it caches and returns the value as a response.

Holder

Holder prevents same HTTP requests to be sent at the same time. Let's assume we have an identifier for a request: {query.foo}. We send a HTTP request /product?foo=bar. While waiting for the response, warden received another HTTP request to the same address which means both HTTP requests are converted to the same key. Then Warden stops the second request. After receiving the response from the first request, Warden returns both requests with the same response by sending only one HTTP request.

Schema

Warden uses custom object -> string transformation to improve performance. Schema will only affect POST requests with json body.

warden.register('test', {
  identifier: '{query.foo}',
  schema: {
    type: 'object',
    properties: {
      name: {
        type: 'string'
      },
      age: {
        type: 'number'
      }
    }
  }
});

warden.request('test', {
  url: 'https://github.com/puzzle-js/puzzle-warden?foo=bar',
  method: 'post',
  json: true,
  body: {
    name: 'Test',
    age: 23
  }
}, (err, response, data) => {
  console.log(data);
})

To enable Schema module, you need to give schema option when registering route. This schema options must be compatible with jsonschema

You should use json: true property.

Retry

When the connection fails with one of ECONNRESET, ENOTFOUND, ESOCKETTIMEDOUT, ETIMEDOUT, ECONNREFUSED, EHOSTUNREACH, EPIPE, EAI_AGAIN or when an HTTP 5xx or 429 error occurrs, the request will automatically be re-attempted as these are often recoverable errors and will go away on retry.

warden.register('routeName', {
  retry: {
    delay: 100,
    count: 1,
    logger: (retryCount) => {
      console.log(retryCount);
    }
  }
}); 

warden.register('routeName', {
  retry: true // default settings
}); 

Default values and properties

| Property | Required | Default Value | Definition | | :--- | :---: | ---: | :--- | | delay | ❌ | 100 | Warden will wait for 100ms before retry | | count | ❌ | 1 | It will try for 1 time by default | | logger | ❌ | | Logger will be called on each retry with retry count|

Debugger

Warden comes with built-in debugger to provide information about request flow.

To enable debug mode:

warden.debug = true;

Flow will be visible on console.

Example log:

4323 | foo_44: HOLDER ---> CACHE

This means the request with the unique id 4323 and identifier value foo_44 is moving from Holder to Cache

Api

warden.register()

Check Registering Route section for better information and usage details

warden.register('routeName', routeConfiguration);

warden.request()

Sends a HTTP request using warden (internally uses request)

warden.request('test', {
  url: `https://postman-echo.com/get?foo=value`,
  method: 'get'
}, (err, response, data) => {
  console.log(data);
});

Any valid property for request module can be used.

warden.requestConfig()

Works exactly like request defaults. It can be used for settings default values for requests.

warden.requestConfig({
  headers: {'x-token': 'my-token'}
});

Sets x-token header with value my-token for all HTTP requests

warden.isRouteRegistered()

Checks whether route is registered.

warden.isRouteRegistered('route'); // true | false

warden.unregisterRoute()

Unregisters route

warden.unregisterRoute('route');

warden.registerCachePlugin()

Registers cache plugin

warden.registerCachePlugin('pluginName', {
  set(){
    
  },
  get(){
    
  }
});

warden.debug

Enables debug mode or disables based on boolean

warden.debug = true;