pouchdb-hypercore

A PouchDB plugin that maps records in hypercores, a P2P append-only datastructure, to a PouchDB or CouchDB instance. The plugin allows you to follow changes in hypercores locally and remotely. In this way, you can build up a database from multiple P2P sources.

As an example, here's how to set up a PouchDB instance to follow a hypercore:

const Hypercore = require('hypercore')
const PouchDB = require('pouchdb')
PouchDB.plugin(require('pouchdb-hypercore'))

async function main () {
  // setup
  const hypercore = Hypercore('.example_hypercore')
  const pouch = new PouchDB('.example_pouchdb')
  await pouch.fromHypercore(hypercore)
  const key = hypercore.key.toString('hex')
  // hypercore -> pouchdb
  const seq = await new Promise((resolve, reject) => {
    this.hyper.append(JSON.stringify({ hello: 'world' }), (err, seq) => {
      if (err) { reject(err) } else { resolve(seq) }
    })
  })
  await new Promise((resolve) => { setTimeout(resolve, 100) })
  const doc = await this.pouch.get(`${key}@${seq}`)
  console.log(doc)
  >>> {_id: '{key}@{seq}', _rev: '1-...', hello: 'world' }
}

Why?

PouchDB and CouchDB are, at this point, enduring and reliable projects with powerful capabilities. Besides their distinctive sync capabilities and efficient map-reduce indexing, CouchDB and PouchDB have vibrant ecosystems of tools, documentation, plugins, and other goodies that reflect the ecosystem's maturity.

The Hypercore protocol has been used to build fascinating P2P projects like Beaker and Cabal. In essence, it gives you mutable torrents by way of a distributed append-only log. In fact it uses the same tracker infrastructure as torrents.

By mirroring hypercores in this way, it becomes easy to produce hypercore-to-http gateways, or to utilize CouchDB's advanced indexing and querying systems.

Why not map changes to writable hypercores?

This feature might return in the future, but for now it is unclear how to do this. Documents produced from hypercore log entries use {key}@{seq} as their ID, but it is impossible to determine that sequence number {seq} before appending the entry to the hypercore. There is also the matter of a hypercore and a PouchDB instance falling out of sync, which further complicates the task.

If you'd like to propose a solution, please chime in with an issue.

Install

Use NPM:

$ npm i @garbados/pouchdb-hypercore

Docs

The plugin adds or modifies the following methods of PouchDB instances:

async pouch.fromHypercore(hypercore)

Streams changes from a hypercore into the database. Does not write changes to the hypercore.

const pouch = new PouchDB(...)
pouch.fromHypercore(hypercore)
  .then(() => { /* ready */ })

async pouch.fromMultifeed(hypercore)

Streams changes from all the hypercores in a multifeed into the database. Does not write changes to the hypercore.

const pouch = new PouchDB(...)
pouch.fromMultifeed(multifeed)
  .then(() => { /* ready */ })

Test

npm test

License

Apache-2.0