@rockset/client

Official Rockset Javascript/Typescript Client

Requirements

Node 10+

Recommended: use Typescript for type checking.

Installation

npm i @rockset/client

Note: you can also install using yarn.

Complete Reference

A complete reference of this library (powered by TypeDoc) can be found here.

Initialization

To initialize a Rockset client, you will have to import the client, and set client parameters.

Import (ES6 Syntax)

Note: you can also import using CommonJS syntax.

import rockset from '@rockset/client';

Configure Client - Apikey

const rocksetClient = rockset(process.env.ROCKSET_APIKEY);

Note: most sans-TypeScript projects currently require the following (we're currently working on better interoperability for Node / CommonJS / ES6):

const rocksetClient = rockset.default(process.env.ROCKSET_APIKEY);

Configure Client - Region

By default, our client connects to the Oregon 1 server. To select another region, you will need to explicitly set this region. You can find a list of Rockset API servers and supported regions in the Rockset Docs here.

const rocksetClient = rockset(process.env.ROCKSET_APIKEY, "https://api.use1a1.rockset.com");

Loading Your Data

Note: we recommend using the Rockset Console for setting up integrations, with which you can create collections that load data from external data sources.

Create an empty collection

rocksetClient.collections
  .createCollection('commons' /* name of workspace */, {
    name: 'MyFirstCollection' /* name of collection */,
    description:
      'I can write data to this collection' /* (optional) description */,
  })
  .then(console.log)
  .catch(console.error);

Create an empty collection with field mappings

import { InputField, OutputField } from '@rockset/client/dist/codegen/api';

rocksetClient.collections
  .createCollection('commons', {
    name: 'Users',
    description: "I'm the map!",
    retention_secs: 10000 /* (optional) number of seconds after which data is purged. */,
  })
  .then(console.log)
  .catch(console.error);

Add documents into your collection with the Write API

rocksetClient.documents
  .addDocuments(
    'commons' /* name of workspace */,
    'Users' /* name of collection */,
    {
      /* array of JSON objects, each representing a new document */
      data: [
        { first_name: 'Brian', last_name: 'Morris', age: '14' },
        { first_name: 'Justin', last_name: 'Smith', age: '78' },
        { first_name: 'Scott', last_name: 'Wong', age: '42' },
      ],
    }
  )
  .then(console.log)
  .catch(console.error);

Query your data

Run a SQL query

rocksetClient.queries
  .query({
    sql: {
      query: 'SELECT * FROM commons.Users u WHERE u.age > :minimum_age',
      /* (optional) list of parameters that may be used in the query */
      parameters: [
        {
          name: 'minimum_age' /* name of parameter */,
          type:
            'int' /* one of: int, float, bool, string date, datetime, time, timestamp */,
          value: '20' /* value of parameter */,
        },
      ],
      default_row_limit: 150 /* (optional) row limit to be used if no limit is specified in the query */,
    },
  })
  .then(console.log)
  .catch(console.error);

Run queries with pagination

let page = await rockset.queries.query({
  sql: {
    // This query will return 100 documents.
    query: 'SELECT * FROM my_collection LIMIT 100',
  },
  // This tells Rockset how many documents to include in the initial response.
  max_initial_results: 20,
});

const qid = page.query_id ?? '';

// `next_cursor` will be null when all queries are read.
while (page.pagination?.next_cursor) {
  // Process the results.
  console.log(page.results);

  page = await rockset.queries.getQueryResults(
    qid,
    page.pagination?.next_cursor ?? '',
    // Read results in batches of 10 results. This can be an arbitrary number.
    10,
    0
  );
}

// Process the last page of results.
console.log(page.results);

Run an async query

let resp = await rockset.queries.query({
  sql: {
    query: 'SELECT * FROM my_collection LIMIT 100',
  },
  // This extends the query timeout to 30 minutes, results wil be available asynchronously
  async: true
});

const qid = resp.query_id ?? '';

let success = false;

while (true) {
  const getQueryResp = await rockset.queries.getQuery(qid);
  const status = getQueryResp.data.status;
  if (status === QueryInfo.StatusEnum.ERROR) {
    console.log('Error', getQueryResp.data.query_errors);
    break;
  }
  else if (status === QueryInfo.StatusEnum.CANCELLED) {
    console.log('Cancelled');
    break;
  }
  else if (status === QueryInfo.StatusEnum.COMPLETED) {
    console.log('Completed');
    let nextCursor = getQueryResponse.data.pagination?.next_cursor;
    // `next_cursor` will be null when all queries are read.
    while (nextCursor) {
      const page = await rockset.queries.getQueryPagination(
        qid,
        nextCursor ?? '',
        // Read results in batches of 10 results. This can be an arbitrary number.
        10,
        0
      );
      console.log(page.results);
      nextCursor = page.pagination?.next_cursor;
    }
  }
}

Run queries using Query Lambdas

// Create a Query Lambda
rocksetClient.queryLambdas
  .createQueryLambda('commons' /* name of workspace */, {
    name: 'MyFirstQueryLambda' /* name of Query Lambda */,
    description: 'A Query Lambda' /* (optional) description */,
    sql: {
      query: 'SELECT * FROM commons.Users u WHERE u.age > :minimum_age',
      /* (optional) list of default parameters that may be used in the query */
      default_parameters: [{ name: 'minimum_age', type: 'int', value: '40' }],
    },
  })
  .then(console.log)
  .catch(console.error);
  
// Run a Query Lambda by tag
rocksetClient.queryLambdas
  .executeQueryLambdaByTag('commons', 'MyFirstQueryLambda', 'latest', {
    /* (optional) list of parameters that may be used in the query, that overwrite default parameters */
    parameters: [{ name: 'minimum_age', type: 'int', value: '20' }],
  })
  .then(console.log)
  .catch(console.error);
  
// Update a Query Lambda
rocksetClient.queryLambdas
  .updateQueryLambda('commons', 'MyFirstQueryLambda', {
    description: 'query for exact age',
    sql: {
      query: 'SELECT * FROM commons.Users u WHERE u.age = :minimum_age',
      default_parameters: [{ name: 'minimum_age', type: 'int', value: '42' }],
    },
  })
  .then((response) => {
    console.log(response);
    
    // Run a Query Lambda by version
    rocksetClient.queryLambdas
      .executeQueryLambda(
        'commons',
        'MyFirstQueryLambda',
        response.data.version /* specific Query Lambda version to run */,
        {
          parameters: [{ name: 'minimum_age', type: 'int', value: '10' }],
          default_row_limit: 150,
          generate_warnings: true,
        }
      )
      .then(console.log)
      .catch(console.error);
      
    // Tag a specific version of your Query Lambda
    rocksetClient.queryLambdas
      .createQueryLambdaTag('commons', 'MyFirstQueryLambda', {
        tag_name: 'dev' /* name of tag */,
        version:
          response.data.version /* specific Query Lambda version to be tagged */,
      })
      .then(console.log)
      .catch(console.error);
  })
  .catch(console.error);

Manage your collections

Create an alias for your collection

rocksetClient.aliases
  .createAlias('commons', {
    name: 'UsersAlias' /* name of alias */,
    description: 'An alias for users collection' /* (optional) description */,
    collections: [
      'commons.Users',
    ] /* list of collection (paths) that your alias references */,
  })
  .then(console.log)
  .catch(console.error);

Query an alias, as you would any collection

rocksetClient.queries
  .query({
    sql: {
      query: 'SELECT * FROM commons.UsersAlias',
    },
  })
  .then(console.log)
  .catch(console.error);

Point an alias to a new collection to avoid downtime

rocksetClient.collections
  .createCollection('commons', {
    name: 'UsersV2',
    description: 'Updated Users collection',
  })
  .then(console.log)
  .catch(console.error);
  
rocksetClient.aliases
  .updateAlias('commons', 'UsersAlias', {
    collections: ['commons.UsersV2'],
  })
  .then(console.log)
  .catch(console.error);

Create a Virtual Instance

rocksetClient.virtualInstances
  .createVirtualInstance({
    name: "analytics",
    type: "LARGE",
    description: "virtual instance for running one-off analytics queries",
    auto_suspend_seconds: 3600,
    mount_type: "LIVE"
  })
  .then(console.log)
  .catch(console.err);;

List Virtual Instances

rocksetClient.virtualInstances
  .listVirtualInstances()
  .then(console.log)
  .catch(console.err);;

Query a Virtual Instance

The virtual instance ID can be found from the listVirtualInstances call.

rocksetClient.virtualInstances
  .queryVirtualInstance("virtual-instance-id", {
    sql: { query: "select 1" },
  })
  .then(console.log)
  .catch(console.err);

Execute a Query Lambda on a Virtual Instance

rocksetClient.queryLambdas
  .executeQueryLambda("commons", "exampleQueryLambda", "version", {
    virtual_instance_id: "virtual-instance-id",
  })
  .then(console.log)
  .catch(console.err);

Testing

Unit tests are available in the tests folder.

Set ROCKSET_APIKEY and ROCKSET_HOST endpoint in the environment variables. To run tests:

yarn test

This runs unit tests and integration tests. You can alternately run individual tests or test files using Jest:

jest src/tests/mockclient.test.ts

Advanced

Custom Fetch Function

By default, the client is a thin wrapper that sends REST calls to Rockset using node-fetch. Some applications may require more complex behavior. If additional functionality is required, the client can be configured to generate the parameters for a REST call, and pass them to a custom fetch function to be handled accordingly.

Here is an example that shows how to support cancelling API calls using a custom fetch function with Axios. To supply a custom fetch function, we pass it in as the last parameter to rockset.

import axios from 'axios';
import rockset from '@rockset/client';

// Simple fetch with Axios instead of node-fetch
const customFetchAxios = async (
  url: string,
  { headers, method, body: data, queryParams: params, cancelToken }: any
) => {
  const res = await axios.request({
    url,
    headers,
    method,
    data,
    params,
    cancelToken,
  });

  return res.data;
};

// Configure
const basePath = 'https://api.rs2.usw2.rockset.com';
const rocksetClient = rockset(
  process.env.ROCKSET_APIKEY,
  basePath,
  customFetchAxios
);
const cancelSource = axios.CancelToken.source();

// Execute a query
rockset.queries
  .query(
    { sql: { query: 'SELECT count(*) FROM _events' } },
    { cancelToken: cancelSource.token }
  )
  .then(console.log)
  .catch(console.error);

// Cancel the request through axios
// Note: this merely cancels the HTTP call - it does not cancel the query on Rockset's servers
cancelSource.cancel();

How to contribute

Please feel free to submit a pull request for modifications that can benefit other users in the community. It is best to have a unit test associated with each pull request.

Support

File any issues with this client through GitHub.

License

This client is licensed under the Apache 2.0 License.