data-point-service
v4.2.9-alpha.0
Published
DataPoint as a Service
Downloads
310
Readme
DataPoint Factory
Creates a DataPoint instance with redis support
Requirements
- Node 8 LTS (or higher)
- Redis (Optional for development)
Install
$ npm install data-point-cache
factory.create
Create a new DataPoint Service Object
factory.create(options):Promise<Service>
Properties of the options
argument:
| option | type | description |
|:---|:---|:---|
| cache | Object
| cache specific settings |
| cache.isRequired | Boolean
| false by default, if true it will throw an error |
| cache.redis | Object
| redis settings passed to the ioredis constructor. For key prefixing, set keyPrefix through cache.redis.keyPrefix
; if the value is not provided then os.hostname() will be used. |
This method returns a Promise that resolves to a Service Object.
The Service Object holds a reference to a dataPoint instance.
const options = {
entities: {
'reducer:foo': (input, acc) => 'bar'
}
}
factory.create(options)
.then((service) => {
return service.dataPoint.transform('reducer:foo')
})
.then((output) => {
console.log(output)
// bar
})
Express example
const express = require('express')
const DataPoint = require('data-point')
const DataPointService = require('data-point-service')
function server (dataPoint) {
const app = express()
app.get('/api/hello-world', (req, res) => {
dataPoint.resolve(`entry:HelloWorld`, req.query)
.then((output) => {
res.send(output)
})
})
app.listen(3000, function () {
console.log('listening on port 3000!')
})
}
function createService () {
return DataPointService.create({
DataPoint,
entities: {
'reducer:HelloWorld': (input, acc) => 'Hello World!!'
}
}).then((service) => {
return service.dataPoint
})
}
createService()
.then(server)
Configure entity's cache settings
To configure an entity's cache settings you must set cache configuration through the params object.
'<type>:<entity-name>': {
params: {
cache: {
ttl: String|Number,
staleWhileRevalidate: String|Number|Boolean,
revalidateTimeout: String|Number
}
}
}
cache.ttl
ttl: String|Number
Use cache.ttl
to set an entity's cache entry Time To Live value. When this value is a string
it is expected to be written using the format supported by ms.
Example:
Creates a request entity that when called will append an entry to redis with a TTL of 20 minutes. When the entry expires, the next request that comes through will have to re-fetch again to respond, upon success a new cache entry with same TTL will be created.
DataPointService.create({
DataPoint,
entities: {
'request:getPlanets': {
url: 'https://swapi.co/api/planets/'
params: {
cache: {
ttl: '20m', // 20 minutes
}
}
}
}
})
cache.staleWhileRevalidate
staleWhileRevalidate: String|Number|Boolean (defaults to `5s` if set to `true`)
staleWhileRevalidate = time to serve stale while revalidating
staleWhileRevalidate
value is expected to be written as a string following the format supported by ms. Alternately it may also be set to true
, which tells the entity to use double the time of the value of its ttl
.
Use cache.staleWhileRevalidate
in conjunction with a valid cache.ttl
to use the Stale While Revalidate cache pattern. When present, caches MAY serve the response in which it appears after it becomes stale, up to the indicated ttl
. The cache SHOULD attempt to revalidate it asynchronously while still serving stale responses. If staleWhileRevalidate
time has passed without the cached entity being revalidated, it SHOULD NOT continue to be served stale.
Example:
Creates a request entity that when called will append an entry to redis with no expiration (stale), at the same time it will create a controller cache entry that has the ttl value. On every request to this entity the stale value will be returned, when the control entry expires, the stale value will continue to be returned and a background process will be triggered to execute the entity and update the stale entry.
DataPointService.create({
DataPoint,
entities: {
'request:getPlanets': {
url: 'https://swapi.co/api/planets/'
params: {
cache: {
ttl: '20m', // 20 minutes
staleWhileRevalidate: true
}
}
}
}
})
cache.revalidateTimeout
revalidateTimeout: String|Number
defaults to: 5s
revalidateTimeout
is the time a revalidation process has before it times-out, the value is expected to be written as a string (eg. '20m'
) following the format supported by ms. When revalidation starts a revalidation flag is set which blocks revalidation duplicates from happening. Once the revalidation times-out the revalidation flag will be removed and the key will be unblocked for being revalidated again. If omitted it defaults to 5 seconds.
This value is only used when staleWhileRevalidate
is also set.
Revalidation and concurrency
When an entity is requested for the first time it will be considered a cold lookup, once the entity is resolved a cache entry will be saved on redis. For the key's life set by the entity's ttl
(and staleWhileRevalidate
) the entity will return the value (considered stale) from the redis store. Once a new request is made for the key and it's ttl
has expired a revalidation will be triggered on the background; when a revalidation starts a revalidation flag will be added to prevent new calls of duplicating the revalidation process.
Revalidation flags are saved locally (in memory - they get cleared once their timeout expires) and also added to the redis store to be accessed by multiple node instances sharing the same keys. The local flag is to prevent duplicate revalidation by a single instance, because it is in memory it is an instant lookup; the remote flag (redis) is to prevent multiple instances of attempting to revalidate the same request.
In the case the revalidation fails the flag will be removed to allow a new revalidation to be triggered.
It is important to know that once the staleWhileRevalidate
delta has also expired a background revalidation will no longer be triggered and cold lookup will be triggered instead. For this reason it is important to carefully set the cache settings to have a solid caching strategy.
Contributing
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
License
This project is licensed under the Apache License Version 2.0 - see the LICENSE file for details