npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

5984

v1.1.0

Published

Lightweight CouchDB CLI and API for efficient stream processing.

Downloads

36

Readme

5984

Lightweight CouchDB CLI and Node API for efficient stream processing.

5984 is a set of a few specific tools. Chaining provides wide flexibility.

5984 batch -n 1000 docs.ndjson \
  | 5984 fetch-revs mydb \
  | 5984 batch -n 100 \
  | 5984 bulk mydb

For now 5984 provides the following commands:

  • bulk-docs: push document to couchdb
  • fetch-revs: fetch document revisions
  • compile: compile couchapp directories and modules
  • batch: batch documents

Please share your thoughts - what is important to you and where you'd see a good fit for it.

The commandline client handles Newline Delimited JSON, which is a perfect fit for CouchDB for many reasons. CouchDB also provides its continuous changes feed as ndjson.

Build Status

Installation

npm install 5984 -g

Interface

You can feed the 5984 CLI either with ndjson files provided via commandline argument or via stdin:

5984 bulk mydb docs.ndjson
echo '{"_id": "mydoc"}' | 5984 bulk mydb

The Node API has a stream interface:

var fnef = require('5984')
var bulk = fnef.bulkDocs('mydb')
bulk.on('data', function (response) {
  console.log(response)
})
bulk.write({ _id: 'mydoc' })

Input Format

Documents can can be given using different formats:

Single Doc

Input chunk can be a single document object:

{ "_id": "one" }
Array of Docs

...or an array of docs:

[
  { "_id": "one" },
  { "_id": "two" }
]
Docs Object

...or an object with a docs property with an array of docs, like eg. used in _bulk_docs requests:

{
  "docs": [
    { "_id": "one" },
    { "_id": "two" }
  ]
}

Common Options

There are some global options:

  • options.url: CouchDB server url (defaults to http://localhost:5984)
  • options.username: username to authenticate with
  • options.password: password to authenticate with

Commands & API

Commands are invoked in a Git-style manner: 5984 <command> [options] Run 5984 without an argument or 5984 help for detailed CLI usage.

The API can be imported either as a whole or directly:

var fnef = require('5984')
var bulkDocs = require('5984/bulk-docs')

All API endpoints return a readable stream in object mode.

bulkDocs(db, options)

Post documents via _bulk_docs to a CouchDB.

  • db: database url. Can also be a database name, in which case the host from options.url is used.

fetchRevs(db, options)

Queries current revisions from database via _all_docs request and inserts them into the documents.

  • db: database url. Can also be a database name, in which case the host from options.url is used.

compile([source], options)

Compile couchapp directories and CommonJS modules. See couchdb-compile for details.

Input is not ndjson but plain newline delimited filenames. If source is provided, use it as input rather than a stream of filenames.

  • source: compile source file. Optional.
  • options.index: look for index.js files. Default is false.

batch(options)

Group documents to batches.

  • options.size: batch size (default is 100)

Examples

Compile design document and push it to the couch:
$ 5984 compile ./ddoc | 5984 fetch-revs mydb | 5984 bulk-docs mydb
{"ok":true,"id":"_design/myapp","rev":"1-967a00dff5e02add41819138abb3284d"}
Create a force push chain with optimized batching:
5984 batch -n 1000 docs.ndjson \
  | 5984 fetch-revs mydb \
  | 5984 batch -n 100 \
  | 5984 bulk mydb

First the input docs are batched into 1000er batches. For each batch a request is being made to retrieve current revisions of the documents. Now the docs are again batched, this time into 100er batches. These batches are posted to CouchDB.

Please also have a look at the tests.

License

Apache 2.0.

(c) 2016 Johannes J. Schmidt