Begin Data

@begin/data

Begin Data is an easy to use, fast, and durable key/value and document store built on top of DynamoDB. Originally built for Begin serverless apps, Begin Data’s core API has three simple methods: get, set, and destroy.

Concepts

Begin Data organizes itself into tables. A table contain documents which are just collections of plain Objects. Documents stored in Begin Data always have the properties table and key.

Optionally a document can also have a ttl property with a UNIX epoch value representing the expiry time for the document.

Usage

Begin Data operates on one DynamoDB table named data with a partition key scopeID and a sort key of dataID (and, optionally, a ttl for expiring documents).

Example app.arc:

@app
myapp

@tables
data
  scopeID *String
  dataID **String
  ttl TTL

Or equivalent CloudFormation YAML:

AWSTemplateFormatVersion: "2010-09-09"
Resources:
    BeginData:
        Type: "AWS::DynamoDB::Table"
        Properties:
            TableName: "data"
            BillingMode: "PAY_PER_REQUEST"
            KeySchema:
              -
                AttributeName: "scopeID"
                KeyType: "HASH"
              -
                AttributeName: "dataID"
                KeyType: "RANGE"
            SSESpecification:
                Enabled: "false"
            TimeToLiveSpecification:
                AttributeName: "ttl"
                Enabled: "TRUE"

Note: projects not based on Architect will need a BEGIN_DATA_TABLE_NAME environment variable. You can also use this env var to override and name the table anything you want. This also allows for multiple apps to share a single table.

API

let data = require('@begin/data')

The core API is three methods:

• data.get(params[, callback]) → [Promise] for retreiving data
• data.set(params[, callback]) → [Promise] for writing data
• data.destroy(params[, callback]) → [Promise] for removing data

Additional helper methods are also made available:

• data.incr(params[, callback]) → [Promise] increment an attribute on a document
• data.decr(params[, callback]) → [Promise] decrement an attribute on a document
• data.count(params[, callback]) → [Promise] get the number of documents for a given table

All methods accept a params object and, optionally, a Node-style errback. If no errback is supplied, a Promise is returned. All methods support async/await.

Writes

Save a document in a table by key. Remember: table is required; key is optional.

let taco = await data.set({
  table: 'tacos',
  key: 'al-pastor'
})

All documents have a key. If no key is given, set will generate a unique key.

let token = await data.set({
  table: 'tokens',
})
// {table:'tokens', key:'LCJkYX9jYWwidW50RhSU'}

Batch save multiple documents at once by passing an Array of Objects.

let collection = await data.set([
  {table: 'ppl', name:'brian', email:'[email protected]'},
  {table: 'ppl', name:'sutr0', email:'[email protected]'},
  {table: 'tacos', key:'pollo'},
  {table: 'tacos', key:'carnitas'},
])

Reads

Read a document by key:

let yum = await data.get({
  table: 'tacos',
  key: 'baja'
})

Batch read by passing an Array of Objects. With these building blocks you can construct secondary indexes and joins, like one-to-many and many-to-many.

await data.get([
  {table:'tacos', key:'carnitas'},
  {table:'tacos', key:'al-pastor'},
])

Destroy

Delete a document by key.

await data.destroy({
  table: 'tacos',
  key: 'pollo'
})

Batch delete documents by passing an Array of Objects.

await data.destroy([
  {table:'tacos', key:'carnitas'},
  {table:'tacos', key:'al-pastor'},
])

Pagination

Large sets of data can not be retrieved in one call because the underlying get api paginates results. In this case use the for await syntax with a limit set to get paginated data.

let pages = data.page({ table:'ppl', limit:25 })
let count = 0  
for await (let page of pages) {
  console.log(page)
  count++
}

Additional Superpowers

• Documents can be expired by setting ttl to an UNIX epoch in the future.
• Atomic counters: data.incr and data.decr

See the tests for more examples!

Patterns

Coming soon! Detailed guides for various data persistence tasks:

• Denormalizing
• Pagination
• Counters
• Secondary indexes
• One to many
• Many to many

More

• Try out Begin Data on Begin!
• Learn more about Begin Data