pagination-stream
v1.2.0
Published
Combine paginated requests in to a single stream
Downloads
1
Readme
pagination-stream
Combine paginated requests in to a single stream
npm install pagination-stream -g
Example
var paginationStream = require('pagination-stream')
var JSONStream = require('JSONStream')
var options = {
urlFormat: 'https://api.github.com/repos/joyent/node/events?page=%d',
headers: {'user-agent': 'pug'},
start: 0,
end: 5,
retries: 2
}
paginationStream(options)
.pipe(JSONStream.parse('*'))
.on('data', function(data) {
console.log(data)
})
paginationStream(opts={})
pagination-stream passes opts
on to request and retry.
See their documentation for all possible options.
opts.urlFormat
- required url string with pagination placeholderopts.end
- required end pageopts.start
- start page, defaults to1
CLI
Usage: pagination-stream [options]
Options:
-u, --url-format <required> URL string with pagination placeholder
-e, --end <required> ending page number
-s, --start starting page number, default is 0
-r, --retries amount of times to retry the operation, default is 10
Example:
$ pagination-stream -u "https://api.github.com/repos/joyent/node/events?page=%d" -s 1 -e 5 -H 'User-Agent: pug' | jsonfilter "*.actor.login"
License
MIT