hubblevector2-sdk-node

The official Hubblevector client for Node.js.

Compatibility

The following table shows the recommended @hubble/hubblevector-sdk-node versions for different Hubblevector versions:

| Hubblevector version | Node sdk version | Installation | | :------------------: | :--------------: | :--------------------------------------- | | v2.2.0+ | latest | yarn add @hubble/hubblevector-sdk-node |

Dependencies

• Hubblevector
• hubble Cloud
• Node: v18+

Installation

You can use npm (Node package manager) or Yarn to install the @hubbledb/hubblevector2-sdk-node dependency in your project:

npm install @hubbledb/hubblevector2-sdk-node
# or ...
yarn add @hubbledb/hubblevector2-sdk-node

This will download the Hubblevector Node.js client and add a dependency entry in your package.json file.

Code Examples

Hubblevector examples

You can find code examples in the examples/hubblevector directory. These examples cover various aspects of working with Hubblevector, such as connecting to Hubblevector, vector search, data query, dynamic schema, partition key, and database operations.

Langchain.js example

You can find a basic langchain.js example in the examples/langchain directory.

next.js example

You can find nextjs app example in the examples/nextjs directory.

Basic usages

This guide will show you how to set up a simple application using Node.js and Hubblevector. Its scope is only how to set up the node.js client and perform the simple CRUD operations. For more in-depth coverage, see the Hubblevector official website.

Start a Hubblevector server

# Download the hubblevector standalone yaml file
$ wget https://github.com/hubblevector-io/hubblevector/releases/latest/download/hubblevector-standalone-docker-compose.yml -O docker-compose.yml

# start the hubblevector server
sudo docker-compose up -d

Connect to Hubblevector

Create a new app.js file and add the following code to try out some basic vector operations using the Hubblevector node.js client. More details on the API reference.

import { HubblevectorClient, DataType } from '@hubbledb/hubblevector2-sdk-node';

const address = 'your-hubblevector-ip-with-port';
const username = 'your-hubblevector-username'; // optional username
const password = 'your-hubblevector-password'; // optional password

// connect to hubblevector
const client = new HubblevectorClient({ address, username, password });

Create a collection

In Hubblevector, the concept of the collection is like the table in traditional RDBMS, eg: mysql or postgres. Before creating a collection, you need to define a schema, then just call the createCollection method.

Define schema for collection

A schema defines the fields of a collection, such as the names and data types of the fields that make up the vectors. More details of how to define schema and advanced usage can be found in API reference.

// define schema
const collection_name = `hello_hubblevector`;
const dim = 128;
const schema = [
  {
    name: 'age',
    description: 'ID field',
    data_type: DataType.Int64,
    is_primary_key: true,
    autoID: true,
  },
  {
    name: 'vector',
    description: 'Vector field',
    data_type: DataType.FloatVector,
    dim: 8,
  },
  { name: 'height', description: 'int64 field', data_type: DataType.Int64 },
  {
    name: 'name',
    description: 'VarChar field',
    data_type: DataType.VarChar,
    max_length: 128,
  },
],

Create the collection

await client.createCollection({
  collection_name,
  fields: schema,
});

Prepare data

The data format utilized by the Hubblevector Node SDK comprises an array of objects. In each object, the key should correspond to the field name defined in the schema. The value type for the key should match the data_type specified in the field of the schema.

const fields_data = [
  {
    vector: [
      0.11878310581111173, 0.9694947902934701, 0.16443679307243175,
      0.5484226189097237, 0.9839246709011924, 0.5178387104937776,
      0.8716926129208069, 0.5616972243831446,
    ],
    height: 20405,
    name: 'zlnmh',
  },
  {
    vector: [
      0.9992090731236536, 0.8248790611809487, 0.8660083940881405,
      0.09946359318481224, 0.6790698063908669, 0.5013786801063624,
      0.795311915725105, 0.9183033261617566,
    ],
    height: 93773,
    name: '5lr9y',
  },
  {
    vector: [
      0.8761291569818763, 0.07127366044153227, 0.775648976160332,
      0.5619757601304878, 0.6076543120476996, 0.8373907516027586,
      0.8556140171597648, 0.4043893119391049,
    ],
    height: 85122,
    name: 'nes0j',
  },
];

Insert data into collection

Once we have the data, you can insert data into the collection by calling the insert method.

await client.insert({
  collection_name,
  fields_data,
});

Ceate index

By creating an index and loading the collection into memory, you can improve the performance of search and retrieval operations in Hubblevector, making it faster and more efficient to work with large-scale datasets.

// create index
await client.createIndex({
  // required
  collection_name,
  field_name: 'vector', // optional if you are using hubblevector v2.2.9+
  index_name: 'myindex', // optional
  index_type: 'HNSW', // optional if you are using hubblevector v2.2.9+
  params: { efConstruction: 10, M: 4 }, // optional if you are using hubblevector v2.2.9+
  metric_type: 'L2', // optional if you are using hubblevector v2.2.9+
});

Hubblevector supports several different types of indexes, each of which is optimized for different use cases and data distributions. Some of the most commonly used index types in Hubblevector include IVF_FLAT, IVF_SQ8, IVF_PQ, and HNSW. When creating an index in Hubblevector, you must choose an appropriate index type based on your specific use case and data distribution.

Load collection

When you create a collection in Hubblevector, the collection data is initially stored on disk, and it is not immediately available for search and retrieval. In order to search or retrieve data from the collection, you must first load the collection into memory using the loadCollectionSync method.

// load collection
await client.loadCollectionSync({
  collection_name,
});

vector search

Now you can perform vector search on your collection.

// get the search vector
const searchVector = fields_data[0].vector;

// Perform a vector search on the collection
const res = await client.search({
  // required
  collection_name, // required, the collection name
  vector: searchVector, // required, vector used to compare other vectors in hubblevector
  // optionals
  filter: 'height > 0', // optional, filter
  params: { nprobe: 64 }, // optional, specify the search parameters
  limit: 10, // optional, specify the number of nearest neighbors to return
  metric_type: 'L2', // optional, metric to calculate similarity of two vectors
  output_fields: ['height', 'name'], // optional, specify the fields to return in the search results
});

Next Steps

• What is Hubblevector
• Hubblevector Node SDK API reference
• Attu, Hubblevector GUI tool, based on Hubblevector node.js SDK
• Feder, anns index visuliazation tool

How to contribute

1. yarn install
2. Fetch hubblevector proto
   1. git submodule init (if this is your first time)
   2. git submodule update --remote
3. Add feature in hubblevector folder.
4. Run test yarn test -- test/Your-test-for-your-feature.spec.ts