Firebolt Node.js SDK

Installation

This library is published in the NPM registry and can be installed using any compatible package manager.

npm install firebolt-sdk --save

# For Yarn, use the command below.
yarn add firebolt-sdk

Using the library

import { Firebolt } from 'firebolt-sdk'

const firebolt = Firebolt();

const connection = await firebolt.connect({
  auth: {
    client_id: process.env.FIREBOLT_CLIENT_ID,
    client_secret: process.env.FIREBOLT_CLIENT_SECRET,
  },
  account: process.env.FIREBOLT_ACCOUNT,
  database: process.env.FIREBOLT_DATABASE,
  engineName: process.env.FIREBOLT_ENGINE_NAME
});

const statement = await connection.execute("SELECT 1");

// fetch statement result
const { data, meta } = await statement.fetchResult();

// or stream result
const { data } = await statement.streamResult();

data.on("metadata", metadata => {
  console.log(metadata);
});

data.on("error", error => {
  console.log(error);
});

const rows = []

for await (const row of data) {
  rows.push(row);
}

console.log(rows)

Contents

• About
• Documentation
• Usage
  • Create connection
    • ConnectionOptions
    • AccessToken
    • Client credentials
    • engineName
  • Test connection
  • Engine URL
  • Execute query
    • ExecuteQueryOptions
    • parameters
    • Named parameters
    • QuerySettings
    • ResponseSettings
  • Fetch result
  • Stream result
  • Result hydration
  • Engine management
    • getByName
    • Engine
      • start
      • stop
  • Database management
    • getByName
    • Database
• Recipes
  • Streaming results
  • Custom stream transformers

About

The Firebolt client for Node.js. firebolt-sdk provides common methods for quering Firebolt databases, fetching and streaming results, and engine management.

firebolt-sdk supports Node.js > v16.

Documentation

Usage

Create connection

const connection = await firebolt.connect(connectionOptions);

ConnectionOptions

type AccessTokenAuth = {
  accessToken: string;
};

type ClientCredentialsAuth = {
  client_id: string;
  client_secret: string;
};

type ConnectionOptions = {
  auth: AccessTokenAuth | ServiceAccountAuth;
  database: string;
  engineName?: string;
  engineEndpoint?: string;
  account?: string;
};

engineName

You can omit engineName and execute AQL queries on such connection.

AccessToken

Instead of passing client id/secret directly, you can also manage authentication outside of node sdk and pass accessToken when creating the connection

const connection = await firebolt.connect({
  auth: {
    accessToken: "access_token",
  },
  engineName: 'engine_name',
  account: 'account_name',
  database: 'database',
});

Client credentials

Default way of authenticating is with the client credentials

const connection = await firebolt.connect({
  auth: {
    client_id: 'b1c4918c-e07e-4ab2-868b-9ae84f208d26',
    client_secret: 'secret',
  },
  engineName: 'engine_name',
  account: 'account_name',
  database: 'database',
});

Token caching

Driver implements a caching mechanism for access tokens. If you are using the same client id/secret for multiple connections, the driver will cache the access token and reuse it for subsequent connections. This behavior can be disabled by setting useCache to false in the connection options.

const connection = await firebolt.connect({
  auth: {
    client_id: 'b1c4918c-e07e-4ab2-868b-9ae84f208d26',
    client_secret: 'secret',
  },
  engineName: 'engine_name',
  account: 'account_name',
  database: 'database',
  useCache: false
});

Test connection

Connection can be tested using:

const firebolt = Firebolt();
await firebolt.testConnection(connectionOptions)

which will perform authentication and a simple select 1 query

Engine URL

Firebolt engine URLs use the following format:

<engine-name>.<account-name>.<region>.app.firebolt.io

For example: your-engine.your-account.us-east-1.app.firebolt.io. You can find and copy your engine endpoint name in the Firebolt web UI.

Execute Query

const statement = await connection.execute(query, executeQueryOptions);

Execute Query with set flags

const statement = await connection.execute(query, {
  settings: { query_id: 'hello' }
});

ExecuteQueryOptions

export type ExecuteQueryOptions = {
  parameters:? unknown[];
  settings?: QuerySettings;
  response?: ResponseSettings;
};

parameters

parameters field is used to specify replacements for ? symbol in the query.

For example:

const statement = await connection.execute("select ?, ?", {
  parameters: ["foo", 1]
});

will produce select 'foo', 1 query

Format Tuple:

import { Tuple } from 'firebolt-sdk'

const statement = await connection.execute("select ? where bar in ?", {
  parameters: [
    1,
    new Tuple(['foo'])
  ]
});

Named parameters

namedParameters field is used to specify replacements for :name tokens in the query.

For example:

const statement = await connection.execute("select :foo, :bar", {
  namedParameters: { foo: "foo", bar: 123 }
});

will produce select 'foo', 123 query

QuerySettings

| Parameter | Required | Default | Description | |---------------|----------|--------------|-----------------------------------| | output_format | | JSON_COMPACT | Specifies format of selected data |

You can also use QuerySettings to specify set flags. For example: { query_id: 'hello' }

ResponseSettings

| Parameter | Required | Default | Description | |-------------------|----------|---------|-------------------------------------------------------| | normalizeData | | false | Maps each row in response from array format to object | | bigNumberAsString | | false | Hydrate BigNumber as String |

Fetch result

const { data, meta, statistics } = await statement.fetchResult();

The Promise API is not recommended for SELECT queries with large result sets (greater than 10,000 rows). This is because it parses results synchronously, so will block the JS thread/event loop and may lead to memory leaks due to peak GC loads.

It is recommended to use LIMIT in your queries when using the Promise API.

Stream result

const { data } = await statement.streamResult();
const rows: unknown[] = [];

data.on("metadata", metadata => {
  console.log(metadata);
});

data.on("error", error => {
  console.log(error);
});

for await (const row of data) {
  rows.push(row);
}

Result hydration

firebolt-sdk maps SQL data types to their corresponding JavaScript equivalents. The mapping is described in the table below:

| Category | SQL type | JavaScript type | Notes | |-------------|----------|-----------------|-----------------------------------------------------------------------------------------------------------------------------------| | Numeric | INT | Number | If value cannot be represented by JavaScript Number (determine using Number.isSafeInteger), BigNumber from "bignumber.js" is used | | | INTEGER | Number | | | | BIGINT | Number | | | | LONG | Number | | | | FLOAT | Number | | | | DOUBLE | Number | | | String | VARCHAR | String | | | | TEXT | String | | | | STRING | String | | | Date & Time | DATE | Date | |

Engine management

Engines can be managed by using the resourceManager object.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const enginesService = firebolt.resourceManager.engine

getByName

Returns engine using engine name.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")

Engine

| Property | Type | Notes | |--------------------------|-------------------------------------------|-------| | name | string | | | endpoint | string | | | current_status_summary | string | |

Start

Starts an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.start()

Stop

Stops an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.stop()

Engine create

Creates an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.create("engine_name");

Attach to database

Attaches an engine to a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.attachToDatabase("engine_name", "database_name");

Engine delete

Deletes an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name");
await engine.delete();

Database management

Databases can be managed by using the resourceManager object.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const databaseService = firebolt.resourceManager.database

Database getByName

Returns database using database name.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name")

Database

| Property | Type | Notes | |---------------|-------------------------------------------|-------| | name | string | | | description | string | |

Database create

Creates a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.create("database_name");

Get attached engines

Get engines attached to a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name");
const engines = database.getAttachedEngines();

Database delete

Deletes a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name");
await database.delete();

Recipes

Streaming results

The recommended way to consume query results is by using streams.

For convenience, statement.streamResult also returns meta: Promise<Meta[]> and statistics: Promise<Statistics>, which are wrappers over data.on('metadata') and data.on('statistics').

const firebolt = Firebolt();

const connection = await firebolt.connect(connectionParams);

const statement = await connection.execute("SELECT 1");

const {
  data,
  meta: metaPromise,
  statistics: statisticsPromise
} = await statement.streamResult();

const rows: unknown[] = [];

const meta = await metaPromise;

for await (const row of data) {
  rows.push(row);
}

const statistics = await statisticsPromise

console.log(meta);
console.log(statistics);
console.log(rows)

Custom stream transformers

To achieve seamless stream pipes to fs or stdout, you can use the Transform stream.

import stream,  { TransformCallback } from 'stream';

class SerializeRowStream extends stream.Transform {
  public constructor() {
    super({
      objectMode: true,
      transform(
        row: any,
        encoding: BufferEncoding,
        callback: TransformCallback
      ) {
        const transformed = JSON.stringify(row);
        this.push(transformed);
        this.push('\n')
        callback();
      }
    });
  }
}

const serializedStream = new SerializeRowStream()

const firebolt = Firebolt();
const connection = await firebolt.connect(connectionParams);
const statement = await connection.execute("select 1 union all select 2");

const { data } = await statement.streamResult();


data.pipe(serializedStream).pipe(process.stdout);

Or use rowParser that returns strings or Buffer:

const { data } = await statement.streamResult({
  rowParser: (row: string) => `${row}\n`
});

data.pipe(process.stdout);

Development process

Actions before

Setup env variables

cp .env.example .env

Execute tests

  npm test

License

Released under Apache License.