5984
v1.1.0
Published
Lightweight CouchDB CLI and API for efficient stream processing.
Downloads
36
Readme
5984
Lightweight CouchDB CLI and Node API for efficient stream processing.
5984 is a set of a few specific tools. Chaining provides wide flexibility.
5984 batch -n 1000 docs.ndjson \
| 5984 fetch-revs mydb \
| 5984 batch -n 100 \
| 5984 bulk mydb
For now 5984 provides the following commands:
- bulk-docs: push document to couchdb
- fetch-revs: fetch document revisions
- compile: compile couchapp directories and modules
- batch: batch documents
Please share your thoughts - what is important to you and where you'd see a good fit for it.
The commandline client handles Newline Delimited JSON, which is a perfect fit for CouchDB for many reasons. CouchDB also provides its continuous changes feed as ndjson.
Installation
npm install 5984 -g
Interface
You can feed the 5984 CLI either with ndjson files provided via commandline argument or via stdin:
5984 bulk mydb docs.ndjson
echo '{"_id": "mydoc"}' | 5984 bulk mydb
The Node API has a stream interface:
var fnef = require('5984')
var bulk = fnef.bulkDocs('mydb')
bulk.on('data', function (response) {
console.log(response)
})
bulk.write({ _id: 'mydoc' })
Input Format
Documents can can be given using different formats:
Single Doc
Input chunk can be a single document object:
{ "_id": "one" }
Array of Docs
...or an array of docs:
[
{ "_id": "one" },
{ "_id": "two" }
]
Docs Object
...or an object with a docs
property with an array of docs, like eg. used in
_bulk_docs
requests:
{
"docs": [
{ "_id": "one" },
{ "_id": "two" }
]
}
Common Options
There are some global options:
options.url
: CouchDB server url (defaults to http://localhost:5984)options.username
: username to authenticate withoptions.password
: password to authenticate with
Commands & API
Commands are invoked in a Git-style manner: 5984 <command> [options]
Run 5984
without an argument or 5984 help
for detailed CLI usage.
The API can be imported either as a whole or directly:
var fnef = require('5984')
var bulkDocs = require('5984/bulk-docs')
All API endpoints return a readable stream in object mode.
bulkDocs(db, options)
Post documents via _bulk_docs
to a CouchDB.
db
: database url. Can also be a database name, in which case the host fromoptions.url
is used.
fetchRevs(db, options)
Queries current revisions from database via _all_docs
request and inserts them
into the documents.
db
: database url. Can also be a database name, in which case the host fromoptions.url
is used.
compile([source], options)
Compile couchapp directories and CommonJS modules. See couchdb-compile for details.
Input is not ndjson but plain newline delimited filenames.
If source
is provided, use it as input rather than a stream of filenames.
source
: compile source file. Optional.options.index
: look forindex.js
files. Default is false.
batch(options)
Group documents to batches.
options.size
: batch size (default is 100)
Examples
Compile design document and push it to the couch:
$ 5984 compile ./ddoc | 5984 fetch-revs mydb | 5984 bulk-docs mydb
{"ok":true,"id":"_design/myapp","rev":"1-967a00dff5e02add41819138abb3284d"}
Create a force push chain with optimized batching:
5984 batch -n 1000 docs.ndjson \
| 5984 fetch-revs mydb \
| 5984 batch -n 100 \
| 5984 bulk mydb
First the input docs are batched into 1000er batches. For each batch a request is being made to retrieve current revisions of the documents. Now the docs are again batched, this time into 100er batches. These batches are posted to CouchDB.
Please also have a look at the tests.
License
Apache 2.0.
(c) 2016 Johannes J. Schmidt