Stable Cache

This lib makes extensive use of a concept called producer. A producer is the source of truth of a cached value. Producers are only called when needed to set a cache, and are usually not needed when there's a cache hit.

The proposal of this library is to make it easier to manage producers and how keys are fetched/saved to redis.

Features:

• there's easily configurable resilience against producer failures:
  • timeout
  • retry with exponential backoff, and decorrelated jitter
  • circuit breaker
• optionally set redis ttl for produced values
• optionally ignoring cached values and forcing values refresh
• optionally refreshing keys in background. Useful when you want keys to be updated before their ttl would evict them.
• optionally producing values in background, while returning the cached value, even on cache misses
• builtin RTA for prometheus

Usage

Cache instance

If the circuitBreaker config is not provided this cache instance simply will not have a circuit breaker flow, and will still be usable.

I recommend to instantiate a cache for each service you consume, or else the circuit breaker would be shared. Which in turn could mean that a failing service would open the circuit for working services too.

const Redis = require('ioredis');
const { Cache } = require('stable-cache');

const redis = new Redis({
  port: 6379,
  host: 'redis', // localhost
});

const cache = new Cache({
  redis, // required to inject a redis client
  options: { // everything is optional from here
    name: 'someService', // name of the service the cache will handle, useful for RTA
    circuitBreaker: { // circuit breaker config for the service
      // percentage of failing producer requests to trigger the circuit breaker
      threshold: 0.2,
      // time window to be considered by the threshold, in milliseconds
      duration: 10000,
      // attempt to half open the circuit after this time in milliseconds
      halfOpenAfter: 20000,
      // don't open the circuit if there's less than this amount of RPS.
      // This avoids opening the circuit, during failures in low load periods.
      minimumRps: 5,
    },
  },
});

Cache.get method

Only the key argument is required. Other options are used to augment the producer flow. See more usage examples at lib/examples/cache-get-usage.js

Cache get options are:

NOTE: circuit breaker policy is configured only when instantiating the Cache class.

| Option | Description | Default Behavior | Default Value | | -: | :- | :- | :- | | producer | A function that will be called when the source of the truth of the key is needed, has the signature function(): Promise<string> | No producer to call | null | | ttl | Time in milliseconds for the redis key ttl, in case the producer resolves | No ttl set | null | | producerRetry | Object configuration for the exponential backoff | No retry | null | | producerRetry.maxDelay | Max amount of delay between retry attempts, in milliseconds | - | 30000 | | producerRetry.maxAttempts | Max amount of retries for the producer | - | 10 | | producerRetry.initialDelay | Initial delay, in milliseconds before the retry | - | 128 | | producerTimeout | Timeout in milliseconds for the producer. Even if there's a timeout, if the producer eventually resolves, the key will be set in background so it would be best if you configure a greater timeout on the producer itself, together with this config | No timeout | null | | returnEarlyFromCache | Whether to return the cached value, and make the producer call on background, on cache miss | Await the producer on cache miss | false | | overrideCache | Whether to ignore cached values and request producer, regardless of cache hits | Don't ignore cache hits, and call producer only if needed | false | | shouldRefreshKey | A callback that if returns true, calls the producer and sets the key on background. Has the signature function(key, currentTTL, options): boolean | No automatic refresh of keys | null |

Example of all the configs:

function producer() {
  return new Promise((resolve) => {
    setTimeout(() => resolve('some-value'), 2000);
  });
}

cache.get(
  'some^key', // required redis key to get
  { // optional configs
    producer,
    ttl: 1000,
    producerRetry: {
      maxDelay: 30000,
      maxAttempts: 10,
      initialDelay: 128
    },
    producerTimeout: 1000,
    returnEarlyFromCache: false,
    overrideCache: false,
    shouldRefreshKey(key, currentTTL, options) {
      if (!options.ttl || currentTTL <= 0) {
        return false;
      }
      // options.ttl is the usually configured ttl for the given key
      const halfLife = options.ttl / 2;
      // refresh key if currentTTL is 50% below the initial ttl
      return currentTTL <= halfLife;
    }
  },
);

Cache.set method

Sets a key with a value, and receives an optional ttl for the key.

Cache set options:

| Option | Description | Default Behavior | Default Value | | -: | :- | :- | :- | | ttl | Time in milliseconds for the redis key ttl | No ttl set | null |

Example:

// sets `some^key` with value `a-value` with ttl of 30 seconds
cache.set(
  'some^key', // required key
  'a-value', // required value
  { // optional configs
    ttl: 30000, // 30 seconds
  }
);

Prometheus Exporter

There's an exporter for prometheus that exposes the following metrics

• cache_results_count: counts cache hits/misses, with labels: ['service', 'operation', 'result']
• cache_operations_duration_seconds: histogram for cache RT, with labels: ['service', 'operation']
• cache_operations_count: counts cache operations made, with labels: ['service', 'operation']
• producer_operations_duration_seconds: histogram with RT for producer calls, with labels: ['service']
• producer_operations_result_count: counts the successes/errors for producer calls, with labels: ['service', 'result']
• producer_circuit_break_state: gauge for the state of a service circuit breaker. A value of 0 means the circuit is closed (working normally), and a value of 1 means the circuit is open (fail fast is activated).

What labels mean?

• service: is the cache name option
• cache operation: type of redis operation, e.g. set, get.
• cache result: cache hit or miss
• producer result: success, or error

Usage

Prometheus exporter options for the constructor are:

| Option | Description | Default Behavior | Default Value | | -: | :- | :- | :- | | prefix | Prefix to be added to every exported metric | No prefix added | '' | | registers | Array of prometheus registers to which metrics should be exported | Use default prom-client register | [Prometheus.register] | | cacheBuckets | Array of numbers, representing the histogram buckets for cache RT | Use default value | Prometheus.exponentialBuckets(0.05, 2, 8) | | producerBuckets | Array of numbers, representing the histogram buckets for producer RT | Use default value | Prometheus.exponentialBuckets(0.1, 2, 8) |

Example:

const Prometheus = require('prom-client');
const { PrometheusExporter } = require('stable-cache');

const exporter = new PrometheusExporter({
  prefix: 'my_app_',
  registers: [Prometheus.register],
  cacheBuckets: Prometheus.exponentialBuckets(0.05, 2, 8),
  producerBuckets: Prometheus.exponentialBuckets(0.1, 2, 8),
});

exporter.collectMetrics(); // start collecting metrics