Stale While Revalidate Cache

This small battle-tested TypeScript library is a storage-agnostic helper that implements a configurable stale-while-revalidate caching strategy for any functions, for any JavaScript environment.

The library will take care of deduplicating any function invocations (requests) for the same cache key so that making concurrent requests will not unnecessarily bypass your cache.

Installation

The library can be installed from NPM using your favorite package manager.

To install via npm:

npm install stale-while-revalidate-cache

Usage

At the most basic level, you can import the exported createStaleWhileRevalidateCache function that takes some config and gives you back the cache helper.

This cache helper (called swr in example below) is an asynchronous function that you can invoke whenever you want to run your cached function. This cache helper takes two arguments, a key to identify the resource in the cache, and the function that should be invoked to retrieve the data that you want to cache. (An optional third argument can be used to override the cache config for the specific invocation.) This function would typically fetch content from an external API, but it could be anything like some resource intensive computation that you don't want the user to wait for and a cache value would be acceptable.

Invoking this swr function returns a Promise that resolves to an object of the following shape:

type ResponseObject = {
  /* The value is inferred from the async function passed to swr */
  value: ReturnType<typeof yourAsyncFunction>
  /**
   * Indicates the cache status of the returned value:
   *
   * `fresh`: returned from cache without revalidating, ie. `cachedTime` < `minTimeToStale`
   * `stale`: returned from cache but revalidation running in background, ie. `minTimeToStale` < `cachedTime` < `maxTimeToLive`
   * `expired`: not returned from cache but fetched fresh from async function invocation, ie. `cachedTime` > `maxTimeToLive`
   * `miss`: no previous cache entry existed so waiting for response from async function before returning value
   */
  status: 'fresh' | 'stale' | 'expired' | 'miss'
  /* `minTimeToStale` config value used (see configuration below) */
  minTimeToStale: number
  /* `maxTimeToLive` config value used (see configuration below) */
  maxTimeToLive: number
  /* Timestamp when function was invoked */
  now: number
  /* Timestamp when value was cached */
  cachedAt: number
  /* Timestamp when cache value will be stale */
  staleAt: number
  /* Timestamp when cache value will expire */
  expireAt: number
}

The cache helper (swr) is also a fully functional event emitter, but more about that later.

import { createStaleWhileRevalidateCache } from 'stale-while-revalidate-cache'

const swr = createStaleWhileRevalidateCache({
  storage: window.localStorage,
})

const cacheKey = 'a-cache-key'

const result = await swr(cacheKey, async () => 'some-return-value')
// result.value: 'some-return-value'

const result2 = await swr(cacheKey, async () => 'some-other-return-value')
// result2.value: 'some-return-value' <- returned from cache while revalidating to new value for next invocation

const result3 = await swr(cacheKey, async () => 'yet-another-return-value')
// result3.value: 'some-other-return-value' <- previous value (assuming it was already revalidated and cached by now)

Configuration

The createStaleWhileRevalidateCache function takes a single config object, that you can use to configure how your stale-while-revalidate cache should behave. The only mandatory property is the storage property, which tells the library where the content should be persisted and retrieved from.

You can also override any of the following configuration values when you call the actual swr() helper function by passing a partial config object as a third argument. For example:

const cacheKey = 'some-cache-key'
const yourFunction = async () => ({ something: 'useful' })
const configOverrides = {
  maxTimeToLive: 30000,
  minTimeToStale: 3000,
}

const result = await swr(cacheKey, yourFunction, configOverrides)

storage

The storage property can be any object that have getItem(cacheKey: string) and setItem(cacheKey: string, value: any) methods on it. If you want to use the swr.delete(cacheKey) method, the storage object needs to have a removeItem(cacheKey: string) method as well. Because of this, in the browser, you could simply use window.localStorage as your storage object, but there are many other storage options that satisfies this requirement. Or you can build your own.

For instance, if you want to use Redis on the server:

import Redis from 'ioredis'
import { createStaleWhileRevalidateCache } from 'stale-while-revalidate-cache'

const redis = new Redis()

const storage = {
  async getItem(cacheKey: string) {
    return redis.get(cacheKey)
  },
  async setItem(cacheKey: string, cacheValue: any) {
    // Use px or ex depending on whether you use milliseconds or seconds for your ttl
    // It is recommended to set ttl to your maxTimeToLive (it has to be more than it)
    await redis.set(cacheKey, cacheValue, 'px', ttl)
  },
  async removeItem(cacheKey: string) {
    await redis.del(cacheKey)
  },
}

const swr = createStaleWhileRevalidateCache({
  storage,
})

minTimeToStale

Default: 0

Milliseconds until a cached value should be considered stale. If a cached value is fresher than the number of milliseconds, it is considered fresh and the task function is not invoked.

maxTimeToLive

Default: Infinity

Milliseconds until a cached value should be considered expired. If a cached value is expired, it will be discarded and the task function will always be invoked and waited for before returning, ie. no background revalidation.

retry

Default: false (no retries)

• retry: true will infinitely retry failing tasks.
• retry: false will disable retries.
• retry: 5 will retry failing tasks 5 times before bubbling up the final error thrown by task function.
• retry: (failureCount: number, error: unknown) => ... allows for custom logic based on why the task failed.

retryDelay

Default: (invocationCount: number) => Math.min(1000 * 2 ** invocationCount, 30000)

The default configuration is set to double (starting at 1000ms) for each invocation, but not exceed 30 seconds.

This setting has no effect if retry is false.

• retryDelay: 1000 will always wait 1000 milliseconds before retrying the task
• retryDelay: (invocationCount) => 1000 * 2 ** invocationCount will infinitely double the retry delay time until the max number of retries is reached.

serialize

If your storage mechanism can't directly persist the value returned from your task function, supply a serialize method that will be invoked with the result from the task function and this will be persisted to your storage.

A good example is if your task function returns an object, but you are using a storage mechanism like window.localStorage that is string-based. For that, you can set serialize to JSON.stringify and the object will be stringified before it is persisted.

deserialize

This property can optionally be provided if you want to deserialize a previously cached value before it is returned.

To continue with the object value in window.localStorage example, you can set deserialize to JSON.parse and the serialized object will be parsed as a plain JavaScript object.

Static Methods

Manually persist to cache

There is a convenience static method made available if you need to manually write to the underlying storage. This method is better than directly writing to the storage because it will ensure the necessary entries are made for timestamp invalidation.

const cacheKey = 'your-cache-key'
const cacheValue = { something: 'useful' }

const result = await swr.persist(cacheKey, cacheValue)

The value will be passed through the serialize method you optionally provided when you instantiated the swr helper.

Manually read from cache

There is a convenience static method made available if you need to simply read from the underlying storage without triggering revalidation. Sometimes you just want to know if there is a value in the cache for a given key.

const cacheKey = 'your-cache-key'

const resultPayload = await swr.retrieve(cacheKey)

The cached value will be passed through the deserialize method you optionally provided when you instantiated the swr helper.

Manually delete from cache

There is a convenience static method made available if you need to manually delete a cache entry from the underlying storage.

const cacheKey = 'your-cache-key'

await swr.delete(cacheKey)

The method returns a Promise that resolves or rejects depending on whether the delete was successful or not.

Event Emitter

The cache helper method returned from the createStaleWhileRevalidateCache function is a fully functional event emitter that is an instance of the excellent Emittery package. Please look at the linked package's documentation to see all the available methods.

The following events will be emitted when appropriate during the lifetime of the cache (all events will always include the cacheKey in its payload along with other event-specific properties):

invoke

Emitted when the cache helper is invoked with the cache key and function as payload.

cacheHit

Emitted when a fresh or stale value is found in the cache. It will not emit for expired cache values. When this event is emitted, this is the value that the helper will return, regardless of whether it will be revalidated or not.

cacheExpired

Emitted when a value was found in the cache, but it has expired. The payload will include the old cachedValue for your own reference. This cached value will not be used, but the task function will be invoked and waited for to provide the response.

cacheStale

Emitted when a value was found in the cache, but it is older than the allowed minTimeToStale and it has NOT expired. The payload will include the stale cachedValue and cachedAge for your own reference.

cacheMiss

Emitted when no value is found in the cache for the given key OR the cache has expired. This event can be used to capture the total number of cache misses. When this happens, the returned value is what is returned from your given task function.

cacheGetFailed

Emitted when an error occurs while trying to retrieve a value from the given storage, ie. if storage.getItem() throws.

cacheSetFailed

Emitted when an error occurs while trying to persist a value to the given storage, ie. if storage.setItem() throws. Cache persistence happens asynchronously, so you can't expect this error to bubble up to the main revalidate function. If you want to be aware of this error, you have to subscribe to this event.

cacheInFlight

Emitted when a duplicate function invocation occurs, ie. a new request is made while a previous one is not settled yet.

cacheInFlightSettled

Emitted when an in-flight request is settled (resolved or rejected). This event is emitted at the end of either a cache lookup or a revalidation request.

revalidate

Emitted whenever the task function is invoked. It will always be invoked except when the cache is considered fresh, NOT stale or expired.

revalidateFailed

Emitted whenever the revalidate function failed, whether that is synchronously when the cache is bypassed or asynchronously.

Example

A slightly more practical example.

import {
  createStaleWhileRevalidateCache,
  EmitterEvents,
} from 'stale-while-revalidate-cache'
import { metrics } from './utils/some-metrics-util.ts'

const swr = createStaleWhileRevalidateCache({
  storage: window.localStorage, // can be any object with getItem and setItem methods
  minTimeToStale: 5000, // 5 seconds
  maxTimeToLive: 600000, // 10 minutes
  serialize: JSON.stringify, // serialize product object to string
  deserialize: JSON.parse, // deserialize cached product string to object
})

swr.onAny((event, payload) => {
  switch (event) {
    case EmitterEvents.invoke:
      metrics.countInvocations(payload.cacheKey)
      break

    case EmitterEvents.cacheHit:
      metrics.countCacheHit(payload.cacheKey, payload.cachedValue)
      break

    case EmitterEvents.cacheMiss:
      metrics.countCacheMisses(payload.cacheKey)
      break

    case EmitterEvents.cacheExpired:
      metrics.countCacheExpirations(payload)
      break

    case EmitterEvents.cacheGetFailed:
    case EmitterEvents.cacheSetFailed:
      metrics.countCacheErrors(payload)
      break

    case EmitterEvents.revalidateFailed:
      metrics.countRevalidationFailures(payload)
      break

    case EmitterEvents.revalidate:
    default:
      break
  }
})

interface Product {
  id: string
  name: string
  description: string
  price: number
}

async function fetchProductDetails(productId: string): Promise<Product> {
  const response = await fetch(`/api/products/${productId}`)
  const product = (await response.json()) as Product
  return product
}

const productId = 'product-123456'

const result = await swr<Product>(productId, async () =>
  fetchProductDetails(productId)
)

const product = result.value
// The returned `product` will be typed as `Product`

Migrations

Migrating from v2 to v3

Return Type

The main breaking change between v2 and v3 is that for v3, the swr function now returns a payload object with a value property whereas v2 returned this "value" property directly.

For v2

const value = await swr('cacheKey', async () => 'cacheValue')

For v3

Notice the destructured object with the value property. The payload includes more properties you might be interested, like the cache status.

const { value, status } = await swr('cacheKey', async () => 'cacheValue')

Event Emitter property names

For all events, like the EmitterEvents.cacheExpired event, the cachedTime property was renamed to cachedAt.

Persist static method

The swr.persist() method now throws an error if something goes wrong while writing to storage. Previously, this method only emitted the EmitterEvents.cacheSetFailed event and silently swallowed the error.

Migrating from v1 to v2

This was only a breaking change since support for Node.js v12 was dropped. If you are using a version newer than v12, this should be non-breaking for you.

Otherwise, you will need to upgrade to a newer Node.js version to use v2.

License

MIT License