scyllo

v1.9.0

Published

2 years ago

The Cassandra/Scylla library you didn't want but got anyways.

Downloads

0High
0Medium
0Low

lucemans

cassandra scylla scyllo database client

./scyllo.png

Installation

Using npm:

npm install scyllo

or if you prefer to use the yarn package manager:

yarn add scyllo

Usage

To start our journey we must first instantiate an instance of our client.

At the heart of Scyllo are the types we create for our tables, to find the appropriate type for a column, check Type Conversion.

Scyllo by default allows you to specify your own type/interface/class without having to bloat your code with any extra information.

type User = {
  user_id: string;
  username: string;
  email: string;
};

type Order = {
  order_id: string;
};

In the below example we instantiate two types, User and Order, we then communicate to Scyllo that we would like to have a table called users and store the User object in there, and a table called orders which stores the Order object. The result will look as follows

{ users: User, orders: Order }

Now we can create the client as follows

const DB = new ScylloClient<{ users: User; orders: Order }>({
  client: {
    contactPoints: ["localhost:9042"], // Where to access the database
    keyspace: "mykeyspace", // Default keyspace
    localDataCenter: "datacenter1", 
  },
});

And voila! All Done! You are ready to Rock and Roll!

Quickstart

All of the above combined results in this final result:

import { ScylloClient } from "scyllo";

type User = {
  user_id: string;
  username: string;
  email: string;
  age: number;
};

type Order = {
  order_id: string;
};

const DB = new ScylloClient<{ users: User; orders: Order }>({
  client: {
    contactPoints: ["localhost:9042"], // Where to access the database
    keyspace: "mykeyspace", // Default keyspace
    localDataCenter: "datacenter1", 
  },
});

Documentation

selectFrom

Selecting from a database table is as easy as it should be. Simply call the selectFrom function on the client instance like so:

const users = await DB.selectFrom("users", "*");

In the event you only want a specific column/field of your object.

const usernames = await DB.selectFrom("users", ["username"]);

In the event you want to apply specific restrictions to what entries should be returned.

const users = await DB.selectFrom("users", ["username"], { user_id: "12345" });

You can also do basic number equality here.

const users = await DB.selectFrom("users", ["username"], { age: eqGreaterThanOrEqual(18) });

Adding extra values to this query should be as simple as

const users = await DB.selectFrom("users", ["username"], { user_id: "12345" }, "ALLOW FILTERING");

selectOneFrom

Selecting a single entry from a table is as easy as it should be. Simply call the selectOneFrom function on the client instance like so:

const user = await DB.selectOneFrom("users", "*");

In the event you only want a specific column/field of your object.

const username = await DB.selectOneFrom("users", ["username"]);

In the event you want to apply specific restrictions to what entries should be returned.

const user = await DB.selectOneFrom("users", ["username"], { user_id: "12345" });

You can also do basic number equality here.

const user = await DB.selectOneFrom("users", ["username"], { age: eqGreaterThanOrEqual(18) });

Adding extra values to this query should be as simple as

const user = await DB.selectOneFrom("users", ["username"], { user_id: "12345" }, "ALLOW FILTERING");

The difference is that the selectOneFrom function will return a single object or undefined instead of an array of objects.

insertInto

Inserting an object into the database is also a breeze. Simply call the insertInto function on the client instance like so:

await DB.insertInto('users', {user_id: "12345", username: "lucemans"});

or simply directly with your user object like so:

const userObject: User = {user_id: "12345", username: "lucemans"};
await DB.insertInto('users', userObject);

Adding extra values to this query should be as simple as

await DB.insertInto('users', {user_id: "12345", username: "lucemans"}, "ALLOW FILTERING");

deleteFrom

Deleting from the database can be done in a multitude of ways. In the event of deleting the entire row, or more of them, from the table, you can do it like so:

await DB.deleteFrom('users', '*', {user_id: "12345", age: eqGreaterThanOrEqual(18)});

The above code will delete the user with the user_id of 12345.

In the event you want to simply delete/clear one of the cells in a specific row you can do that like so:

await DB.deleteFrom('users', ['username'], {user_id: "12345"});

Adding extra values to this query should be as simple as

await DB.deleteFrom('users', ['username'], {user_id: "12345"}, "ALLOW FILTERING");

update

Update allows you to edit only some parts of a row. This is done by passing an object with the values you want to update and the criteria to find the row you want to update.

await DB.update('users', {username: "lucemans"}, {user_id: "12345", age: eqGreaterThanOrEqual(18)})

Adding extra values to this query should be as simple as

await DB.update('users', {username: "lucemans"}, {user_id: "12345"}, "ALLOW FILTERING");

createTable

Creating a table is easily done with the createTable function.

await DB.createTable('users', 
    true, // Wether we should create the table if it does not exist yet. Will throw an error if the table already exists.
    {
        user_id: { type: "text" },
        username: { type: "text" },
    },
    'user_id' // The primary key for the table.
);

The primary key can also be a composite key if you want a combination of multiple keys to be the primary key.

await DB.createTable('users', 
    true,
    {
        user_id: { type: "text" },
        username: { type: "text" },
    },
    ['user_id', 'username'] // The composite key for the table.
);

Additionally a clustering-key can also be provided as follows

await DB.createTable('users', 
    true,
    {
        user_id: { type: "text" },
        username: { type: "text" },
    },
    'user_id', // The primary key for the table.
    'username' // The clustering key for the table.
);

dropTable

Dropping a table is a very simple process.

await DB.dropTable('users');

truncateTable

Truncating a table is a very simple process as well.

await DB.truncateTable('users');

createIndex

Sometimes you will want to index a table by different fields. So that you can search your user table by both user_id and email for example. In scylla we use Secondary Indexes for this. You can create them like so:

await DB.createIndex('users', 'users_by_email', 'email');

Once the key is created we are able to query the same users table as follows

await DB.selectOneFrom('users', '*', { email: '[email protected]' });

createLocalIndex

In contrary to createIndex, createLocalIndex allows you to create a narrower search ability, thus letting you optimize your database performance even more. Usng localIndex you can search your user table by both user_id and user_id and email for example. To read more on localIndexes, have a look at Local Secondary Indexes for this. Regardless You can create them like so:

await DB.createLocalIndex('users', 'users_by_email', 'email');

Once the key is created we are able to query the same users table as follows

await DB.selectOneFrom('users', '*', { user_id: 'lucemans', email: '[email protected]' });

Do keep in mind that usage of local indexes do require you to know the primary key during query-time. If you are looking to search only on the secondary key, try createIndex.

useKeyspace

If you want to use another keyspace than the one specified in the client options, you can use the useKeyspace function.

await DB.useKeyspace('myotherkeyspace');

This will error out if the keyspace doesn't exist. To get around this, you could pass in a boolean as the second argument.to create the keyspace if it doesn't exist.

await DB.useKeyspace('myotherkeyspace', true);

createKeyspace

Using createKeyspace you can create a keyspace. This function is useful in situations when we want to be more specific about our keyspace settings.

A simple keyspace:

await DB.createKeyspace('mykeyspace');

A keyspace with replicationClass and replicationFactor

await DB.useKeyspace('myotherkeyspace', 'SimpleStrategy', 1);

dropKeyspace

Dropping a keyspace is a very simple process.

await DB.dropKeyspace('mykeyspace');

By default, it will only drop the keyspace if it exists. If you want to try to drop the keyspace without IF EXISTS, then you can pass in a boolean as the second argument. Note that this will throw an error if the keyspace doesn't exist.

await DB.dropKeyspace('mykeyspace', false);

awaitConnection

In some cases you will want to trigger database requests in order to run queries right after. Or maybe you just want to be 100% sure we can connect to the database.

We can use awaitConnection for this!

await DB.awaitConnection();

The above code snippet will simply wait for the database to have fully booted up and connected and then allow further code to execute.

shutdown

The shutdown function is simple and straight forward. It shuts down the client. Although it might not always be necessary it is probably a good idea to shutdown the connection when you exit your process.

await DB.shutdown();

raw / rawWithParams

In the event you would like to execute a CQL query that Scyllo does not support yet, thats totally okay too! Use the .raw function in order to do so

await DB.raw('GRANT SELECT ON ALL KEYSPACES TO lucemans');

If the query you are trying to execute requires injecting javascript-objects then you could do the following

await DB.rawWithParams(
  'INSERT INTO users (user_id, username, email) VALUES (?, ?, ?)',
  [Long.fromString('987654321'), 'lucemans', '[email protected]']
);

The above with attempt to safely convert the javascript objects into Cassandra friendly data in order to keep everything working.

query

The query function is similar to the rawWithParams function as mentioned above and allows for an object-way of inserting values.

await DB.query({
  query: 'INSERT INTO users (user_id, username, email) VALUES (?, ?, ?)',
  args: [Long.fromString('987654321'), 'lucemans', '[email protected]']
});

batch

The batch function allows you to execute multiple queries simultaneously. This can save bandwidth and the time it takes for those queries to finish.

await DB.batch()
        .insertInto('users', { username: 'Antony', uid: 123456000 })
        .insertInto('users', { username: 'Luc', uid: 123456001 })
        .update('users', { username: "Lucemans" }, { uid: 123456001 })
        .execute()

Note: This function only supports INSERT, UPDATE and DELETE queries.

migrations

Migrations are an opinionated way of managing database structure changes through development.

type DBType = { user: { id: number }, something: { another_id: number } };

const migration1: Migration<DBType> = async (database) => {
  await database.createTable(
    'user',
    true,
    { id: { type: 'int' } },
    'id'
  );
}

// doesn't need to be exact DBType
const migration2: Migration<Pick<DBType, "something">> = async (database, log) => {
  await database.createTable(
    'something',
    true,
    { another_id: { type: 'int' } },
    'another_id'
  );

  log("message from migration");
}

// NOTE: please, please, please, please add migrations one after another and not in the middle
//       ...it breaks stuff
await DB.migrate([
  migration1,
  migration2
], true); // true enables logging and uses the log function passed to client

What to use for ID's

However nice Cassandra/Scylla and hereby also Scyllo may be, ID generation is not something native to the database. This is done by design, see Lightweight Transactions. When looking at companies that utalise CQL at scale, we notice the common trend of Snowflake ID, a Twitter created system for generating IDs at scale.

Obviously @v3xlabs has a library for this, and we highly recommend you checkout Sunflake, a zero-depedency lightweight typescript-safe Snowflake ID generator.

Debug Mode & Logging

In the event you encounter situations where you would like to view the underlying queries, scyllo allows you to set debug-mode.

Simply during creation of the client, specify debug as `true.

const DB = new ScylloClient<{ users: User; orders: Order }>({
  client: {
    contactPoints: ["localhost:9042"],
    keyspace: "mykeyspace",
    localDataCenter: "datacenter1",
  },
  debug: true // <-- Enable Debug Mode
});

Debug Mode by default will log all performed queries to the console. If you would rather have these queries piped to some other method (@lvksh/logger in our example) then you can do that with the following.

import { ScylloClient } from 'scyllo';
import { createLogger } from '@lvksh/logger';

// Initialize the logger
const logger = createLogger({
  database: 'DB',
  info: 'INFO',
  success: 'OK'
});

// Initialize the database
const DB = new ScylloClient<{ users: User; orders: Order }>({
  client: {
    contactPoints: ["localhost:9042"],
    keyspace: "mykeyspace",
    localDataCenter: "datacenter1",
  },
  debug: true,
  log: logger.database // <-- use the logger.database function
});

Type Conversion

In order to keep track of the corresponding Javascript type for a CQL data type, we can use the link bellow. https://docs.datastax.com/en/developer/nodejs-driver/4.6/features/datatypes/

When prepare = true (default behaviour) in the Scyllo config, it understand that a javascript types should be converted to the corresponding cassandra type. When prepare = false it may cause issues with more advanced types as it will not convert them, it thus recommended to leave this option as true.

Note that Tuples have to be create in a specific way, which you can read about here.

Known Restrictions

Limited multi-keyspace support. This is something that is on our roadmap and hopefully some day in the future we will be able to handle this. As of right now scyllo is unable to provide you with a smooth multi-keyspace experience, but this is in the works.

Contributors

LICENSE

This package is licensed under the GNU Lesser General Public License.

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

Table of Contents

Installation

Usage

Quickstart

Documentation

selectFrom

selectOneFrom

insertInto

deleteFrom

update

createTable

dropTable

truncateTable

createIndex

createLocalIndex

useKeyspace

createKeyspace

dropKeyspace

awaitConnection

shutdown

raw / rawWithParams

query

batch

migrations

What to use for ID's

Debug Mode & Logging

Type Conversion

Known Restrictions

Contributors

LICENSE