@venzee/dynamo_streams

NodeJs Stream wrappers around DynamoDB requests.

Installation

npm i -S @venzee/dynamo_streams

Usage

QueryStream

You can inject DocumentClient instances or DynamoDB instances. The stream uses calls the query method of the injected client with the provided params. The stream handles pagination and will emit all items found that match the provided query.

See the documentation for DynamoDB and DocumentClient for more info;

const client = new AWS.DynamoDB.DocumentClient();
const queryStream = require( '@venzee/dynamo_streams/query' )( client, params );

queryStream
  .on( 'data', doStuffWithData)
  .on( 'error', handleError);

ScanStream

Same behavior as a QueryStream, except we are streaming scan results, not query results.

Errors

When query or scan return errors, the 'error' event raised by the stream includes the error raised by the method as well as the parameters used to last invoke the method. This allows client code to retry using custom retry logic, for example.

function runQuery( params, doStuffWithData, onQueryRun ){

  const client = new AWS.DynamoDB.DocumentClient();
  const queryStream = require( '@venzee/dynamo_streams/query' )( client, params );

  queryStream
    .on( 'data', doStuffWithData)
    .on( 'error', onQueryRun )
    .on( 'finish', onQueryRun );
}

function onQueryRun( err ){

  if( err ){

    const { error, queryParameters } = err;

    if( error.code === 'ProvisionedThroughputExceededException' ) return setTimeout( runQuery, 1000, queryParameters, doStuffWithData, onQueryRun );
    return handleError( error );

  }

}