pull-write
v1.1.4
Published
base class for creating generic pull-sinks that write to some device via an async call.
Downloads
1,044
Readme
pull-write
base class for creating generic pull-sinks that write to some device via an async call.
Write(asyncWrite, reduce, max, cb)
asyncWrite(ary, cb)
async function called with an array of items to output.
This function will only ever be called once at a time (per instance),
and while it is working pull-write
will buffer any subsequent writes,
until the buffer has the length of at most max
,
or asyncWrite
has called back.
reduce (queue, item)
queue
is the current backlog of data the pull-write
is getting ready to write.
item
is the next incoming item. reduce
must add item
into queue
in whatever way is appropiate. If queue
is empty, then it will be null
.
Your reduce
function must handle that case and set an initial value.
by default, reduce
will be a function that initializes a buffer,
and then pushes the new items onto that buffer, this means max
will be
compared to the number of items in that buffer.
max
A number, when the .length
property of the queue
returned by reduce
gets this big pull-write
will stop reading more, until asyncWrite
calls back.
example
Suppose we want a to take a stream of values from one leveldb, and write it to another. If we have the timestamp they where written to the first, we can track that in the second, then it's easy to keep them both up to date. We just need to always output latest ts separately.
var Write = require('pull-write')
var LevelWrite = function (db, cb) {
var max = 100
return Write(function (ary, cb) {
db.batch(ary, cb)
}, function (queue, data) {
if(!queue)
queue = [{key: '~meta~ts', value: 0, type: 'put'}]
queue.push({key:data.key, value: data.value, type: 'put'})
//the record of the current sequence is always the first value
//in the batch, so we can update it easily.
queue[0].value = data.ts
return queue
}, max, cb)
}
License
MIT