Squirrel

Local mirror mecanism for ETCD

Keep a replication of ETCD folder locally for low latency querying. Provide an index system to access a file without scanning all nodes.

Summary

• Install
• Usage
  • Node Interface
    • get(path)
    • getBy(index, key)
    • getAll(index)
    • set(path, value)
  • Command Line Interface
    • squirrel-sync
    • squirrel-watch
• Index System
• Fallback System
• Test

Install

$ npm install --save @coorpacademy/squirrel

import createSquirrel from '@coorpacademy/squirrel';

Usage

Node Interface

const squirrel = createSquirrel({
    hosts: 'http://localhost:2379',
    auth: null,
    ca: null,
    key: null,
    cert: null,

    cwd: '/',
    fallback: '/tmp/squirrel.json',
    indexes: ['foo', 'bar.baz']
});

Options:

• hosts: ETCD hosts. more
• auth: A hash containing {user: "username", pass: "password"} for basic auth. more
• ca: Ca certificate. more
• key: Client key. more
• cert: Client certificate. more
• cwd: ETCD current working directory.
• fallback: Temporary file to save ETCD backup.
• indexes: Array of key to index.

Methods

Consider the following folder:

/
├── bar
│   └── baz   { "bar": { "baz": "qux" } }
└── foo       { "foo": "bar" }

get(path)

Get file by path. Returns Promise;

• path (String): the path of the file to get.

const foo = await squirrel.get('/foo');
console.log(foo); // { "foo": "bar" }

const barBaz = await squirrel.get('/bar/baz');
console.log(barBaz); // { "bar": { "baz": "qux" } }

getBy(index, key)

Get by index value. Returns Promise;

• index (String): the path of the property to get. It needs to be declared in the indexes option
• key (String): the value to match

const foo = await squirrel.getBy('foo', 'bar');
console.log(foo); // { "foo": "bar" }

const barBaz = await squirrel.getBy('bar.baz', 'qux');
console.log(barBaz); // { "bar": { "baz": "qux" } }

Fields can be nested, as described by _.get.

getAll(index)

Get index Map. Returns Promise;

• index (String): the path of the property to get. It needs to be declared in the indexes option

const foo = await squirrel.getAll('foo');
console.log(foo); // { "bar": { "foo": "bar" } }

const barBaz = await squirrel.getAll('bar.baz');
console.log(barBaz); // { "qux": { "bar": { "baz": "qux" } } }

set(path, value)

Set file by path. Returns Promise;

• path (String): the path of the file to get.
• value (Object): An object to store in file. Will be serialized.

const foo = await squirrel.set('/foo',  { "foo": "bar" });
console.log(foo); // { "foo": "bar" }

Command Line Interface

squirrel-sync

Synchronize FS folder with ETCD folder.

$ squirrel-sync --hosts localhost:2379 ./fs-folder /etcd-folder

squirrel-watch

Watch ETCD folder changes.

$ squirrel-watch --hosts localhost:2379 /etcd-folder

squirrel-dump

Write ETCD folder in preloadedStore format.

$ squirrel-dump --hosts localhost:2379 /etcd-folder ./dump.json

Arguments

• --hosts="host1,host2": ETCD hosts. more
• --ca=/file.ca: Ca certificate. more
• --key=/file.key: Client key. more
• --cert=/file.cert: Client certificate. more

Index System

Squirrel allows to put JSON in file. In this case, it could be indexes to access directly. Consider the following ETCD directory.

/
├── file1 { "foo": "bar" }
├── file2 { "foo": "baz" }
└── file3 { "foo": "qux" }

First of all, we should indicate Squirrel which paths we want to index.

const squirrel = createSquirrel({
  indexes: ['foo']
});

Now, we can get the contents of file1 by searching for its foo value.

const file1 = await squirrel.getBy('foo', 'bar');
console.log(file1); // { "foo": "bar" }

We can also get the value of the index as an object.

const fooIndex = await squirrel.getAll('foo');
console.log(fooIndex);
/*
{
  "bar": { "foo": "bar" },
  "baz": { "foo": "baz" },
  "qux": { "foo": "qux" }
}
 */

If two files have the same index value, Squirrel keeps one of the two.

Squirrel scans all files, no matter how deep, that contain a JSON value.

Index could be a complex path, as long as it works with _.get.

Fallback System

By declaring a fallback path, Squirrel is able :

• to save its state every time a change is made
• to restore the state to be faster on the next restart even if ETCD isn't available.

Test

You may run tests with

$ npm test

Marble