id: tutorial-store title: Wireline Key/Value Store

A simple key/value storage service. Supported operations are:

• set
• get
• inc
• del
• scan
• clear
• format

Quick Start

Add the Wireline key/value store API client to your project:

$ yarn add @wirelineio/store-client

Then import it in your code and create the Store client object by passing it the context:

import Wireline from '@wirelineio/sdk';
import Store from '@wirelineio/store-client';

module.exports = {
  test: Wireline.exec(async (event, context) => {
    let store = new Store(context);

    // The rest of your code here...
  })
};

The last step is to indicate which deployment of the wireline.io/store service you would like to use, either a new one or an existing one. The HTTP endpoint details for whichever wireline.io/store you choose will be infused automatically into the context object by Wireline.exec(), providing everything needed to configure your Store client object.

In this example, we will use one that already exists by adding it as a reference to our stack.yml. Use the wire CLI to list active deployments of the wireline.io/store service.

$ wire svc list-deployments --service 'wireline.io/store'
# NOTE: Some columns omitted for clarity.

Domain        Stack         Service               WRN                                 Version
---------------------------------------------------------------------------------------------
wireline.io   wireline      wireline.io/store     wrn::wireline.io/wireline/store     0.0.1
...

Add a reference to it at the bottom of the stack.yml of your Wireline project using its name, wireline.io/store, and the WRN of the deployment you wish to use, wrn::wireline.io/wireline/store.

  wireline.io/store:
    reference:
      endpointUrl: wrn::wireline.io/wireline/store

Get/Set

The most basic operations are set and get, storing or retrieving a key/value pair (respectively). The key must be a string. The value may be a string, number. boolean, or an object.

    await store.set('key', 'value'));
    let value = await store.get('key');

Inc

This provides an atomically incrementing counter. If the counter does not exist, it will be created and set to a value of 1.

    let value = await store.inc('counterA');

Del

Removes a single key/value pair, if it exists. No value is returned, and there is no indication as to whether the key existed.

    await store.del('counterA');

Scan

Scan for key/value pairs in the bucket. If provided, prefix restricts the results to keys which equal or begin with that prefix. This is a case-sensitive comparison. The limit option will limit the maximum number of results returned. Depending on the number of pairs and the storage provider, this may be an expensive operation.

  # Scan everything.
  let results = await store.scan();

  # Scan for up to ten items, with keys that equal or start with 'aaa'.
  results = await store.scan('aaa', { 'limit': 10 })

Clear

Clear all key/value pairs in the bucket. Depending on the number of pairs and the storage provider, this may be an expensive operation.

  await store.clear();

Format

Clear all key/value pairs in all buckets, as well as removing the "stake". Depending on the storage provider, this may be an extremely expensive operation.

Using Buckets

Nearly all operations support the bucket option, which scopes the operation to the specified "bucket". A "bucket" provides namespacing for keyi/value pairs, so that keyA in bucketA is distinct from keyA in bucketB.

  store.set('keyA', 'valueA', { 'bucket': 'bucketA' });

The default bucket may also be set when the Store client object is created:

  let store = new Store(ctx, { 'bucket': 'myBucket' });

Planting Stakes

Each Wireline deployment is given a unique namespace within the store. The first operation for the deployment implicitly "stakes a claim" to the namespace by generating a unique value which will be used to validate all subsequent operations. In addition to the unique Wireline WRN details (domain/stack/deployment), the account ID performing the operation is tied to the "stake". If multiple accounts are used, an explicit "stake" value should be used instead. This can be provided when creating the Store client object. The value should be treated securely, as a kind of credential.

  let store = new Store(ctx, { 'stake': 'myVerySecretStakeValue' });

The format operation not only removes all values from the unique namespace given to the deployment, it releases the stake.

Configuration

The HTTP endpoint will be automatically discovered by adding a reference to a deployed store's WRN under the name wireline.io/store in either the stack.yml or wireline.yml for your service. If for some reason that automatic process needs to be overridden, the full HTTP URL can be set explicitly using the WRL_STORE_URL environment variable.

stack.yml WRN:

name: my-stack
domain: example.com
stage: dev
provider:
  name: aws
  region: us-east-1
stack:
  my-deployment:
    deployment:
      image: 'wrn::my-service'
      version: 0.0.1
  wireline.io/store:
    reference:
      endpointUrl: 'wrn::wireline.io/wireline/store'

wireline.yml WRN:

services:
  'wireline.io/store': 'wrn::wireline.io/wireline/store'

stack.yml by ENV:

name: my-stack
domain: example.com
stage: dev
provider:
  name: aws
  region: us-east-1
stack:
  my-deployment:
    deployment:
      image: 'wrn::my-service'
      version: 0.0.1
      environment:
        WRL_STORE_URL: 'https://example.store.url/store'

Example Usage

Example A - Basic functionality

    let store = new Store(context);

    // All of these operations will happen in the default bucket.
    console.log(await store.set('A-key', 'valA'));
    console.log(await store.get('A-key'));

    console.log(await store.inc('A-counter'));
    console.log(await store.inc('A-counter'));
    console.log(await store.inc('A-counter'));

    console.log(await store.scan('A-'));

    await store.del('A-key');

    console.log(await store.scan('A-'));

    await store.clear();
    console.log(await store.scan());

Example B - Using different buckets to separate types and indices...

    let store = new Store(context);
    
    let userOpts = { 'bucket': 'users' };
    let idxUserByEmailOpts = { 'bucket': 'idx.users.email' };
    let idxUserByUsernameOpts = { 'bucket': 'idx.users.username' };
    
    let deviceOpts = { 'bucket': 'devices' };
    let idxDevByUser = { 'bucket': 'idx.devices.byuser', limit: 50 };
    
    // Insert 'bob' by his UUID.
    await store.put(bob.uuid, bob, userOpts);
    
    // Index bob's username to his UUID.
    await store.put(bob.username, bob.uuid, idxUserByUsernameOpts);
    
    // Also index bob's e-mail address to his uuid.
    await store.put(bob.email, bob.uuid, idxUserByEmailOpts);
    
    // Insert bob's device into the device's bucket.
    await store.put(dev.uuid, dev, deviceOpts);
    
    // And index it to his user.
    await store.put(`${bob.uuid}::${dev.uuid}`, { 'usr:' bob.uuid, 'dev': dev.uuid }, idxDevByUser);
    
    // Scan for all bob's devices...
    let results = await store.scan(`${bob.uuid}::`, idxDevByUser);