In-Memory Vector Embeddings Database

BETA

This project provides an in-memory vector embeddings database using embeddings for efficient querying and searching of text documents.

Description

The project allows you to create, query, and manage an in-memory vector embeddings database. It uses embeddings to represent text documents as vectors, enabling efficient similarity searches.

Installation

To install the package, use the following command:

npm install @j-o-r/vdb

Usage

Here is an example of how to use the Vdb class to create a database, perform searches, and manage the database:

import path from 'path';
import Vdb from '@j-o-r/vdb';

const db = new Vdb('path/to/storage');

// Create a database from a text document
// You only have to do this ones.
// It may take some time
if (!db.list().includes('readme')) {
  const file = path.resolve('README.md');
  await db.create(file, 'readme');
}
// Perform a search in the database
let str = await db.search('readme', 'How to create a database', { treshhold: 0.86,  results: 4, preRead: 1, postRead: 10 });
console.log(str);

// -- Delete a database
// db.delete('readme');

API

Vdb Class

Constructor

new Vdb(storagePath)

• storagePath (string): Path to the storage folder.

Methods

• list(): Returns a list of available databases.
• delete(dbName): Deletes the specified database.
  • dbName (string): Name of the database to delete.
• create(file, dbName, batchSize): Creates or overwrites an embeddings database from a text document.
  • file (string): Path to the text document.
  • dbName (string): Name of the database.
  • batchSize (number, optional): Batch size for processing (default is 256).
• search(dbName, query, selector): Searches the database and returns formatted results.
  • dbName (string): Name of the database.
  • query (string): Search query.
  • selector (object, optional): Selector options.
    • results (number): Number of results to return.
    • preRead (number): Number of lines to return before the found index.
    • postRead (number): Number of lines to return after the found index.
• getResult(dbName, query, results): Gets raw search results from the database.
  • dbName (string): Name of the database.
  • query (string): Search query.
  • results (number, optional): Number of results to return (default is 5).

License

This project is licensed under the APACHE 2.0 License. See the LICENSE file for details.