SingleStore RAG

A module that enhances the @singlestore/client package with Retrieval-Augmented Generation (RAG) functionality, enabling seamless integration of advanced RAG features into a chat application.

Table of Contents

• Installation
• Example Apps
• Usage
  • Initialization
    • Additional Notes
  • RAG
    • Get Models
  • Chat
    • Create Chat
      • Additional Notes
    • Find Chat
      • All Chats
      • By Condition
      • With Advanced Filtering
    • Update Chat
      • By Condition
      • Specific
    • Delete Chat
      • By Condition
      • Specific
  • Chat Session
    • Create Chat Session
    • Find Chat Session
      • All Chat Sessions
      • By Condition
      • With Advanced Filtering
    • Update Chat Session
      • By Condition
      • Specific
    • Delete Chat Session
      • By Condition
      • Specific
    • Submit
  • Chat Message
    • Create Chat Message
    • Find Chat Message
      • All Chat Messages
      • By Condition
      • With Advanced Filtering
    • Update Chat Message
      • By Condition
      • Specific
    • Delete Chat Message
      • By Condition
      • Specific
  • File
    • Upload File
    • Delete File
  • Tools

Installation

npm install @singlestore/rag @singlestore/ai @singlestore/client

Example Apps

• Terminal Chat

Usage

Initialization

To initialize RAG, you need to create instances of the AI class from @singlestore/ai and the SingleStoreClient from @singlestore/client. After that, establish a connection to your workspace and select the database where the required tables for the RAG application will be created to store chats, chat sessions and chat messages.

import { AI } from "@singlestore/ai";
import { SingleStoreClient } from "@singlestore/client";
import { RAG } from "@singlestore/rag";

const ai = new AI({ openAIApiKey: "<OPENAI_API_KEY>" });
const client = new SingleStoreClient({ ai });

const connection = client.connect({
  host: "<WORKSPACE_HOST>",
  user: "<WORKSPACE_USER>",
  password: "<WORKSPACE_PASSWORD>",
});

const database = connection.database.use("<DATABASE_NAME>");
const rag = new RAG({ database, ai });

Additional Notes

• It is possible to use custom LLMs instead of OpenAI. For more information, please refer to this guide.

RAG

Get Models

Returns the list of available models that can be used in the ChatSession.submit method.

const models = await rag.getModels();

Chat

Create Chat

Creates a chat where sessions and messages will be stored.

const chat = await rag.chat.create({
  name: "<CHAT_NAME>", // Optional
  systemRole: "<SYSTEM_ROLE>", // Optional
  tableName: "<TABLE_NAME>", // Optional; `chats` by default
  sessionsTableName: "<SESSIONS_TABLE_NAME>", // Optional; `chat_sessions` by default
  messagesTableName: "<MESSAGES_TABLE_NAME>", // Optional; `chat_messages` by default
  store: true, // Optional; `false` by default,
  tools: toolsList, // Optional
});

Additional Notes

• The store parameter determines whether the chat, sessions, and messages should be persisted in the database.
• The tools parameter accepts an array of tools that the LLM can use when executing the ChatSession.submit method. For more information, please refer to this guide.

Find Chat

Retrieves chat records, either all available chats or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.

All Chats

Fetches all chat records without any filtering.

const chats = await rag.chat.find();

By Condition

Fetches a specific chat record based on a provided condition, such as id.

const chat = await rag.chat.find({ where: { id: "<ID>" } });

With Advanced Filtering

Fetches chat records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset), and provide custom parameters such as tableName and tools.

const chats = await rag.chat.find(
  {
    where: { columnName: "COLUMN_VALUE" }, // Optional
    orderBy: { columnName: "asc" }, // Optional
    limit: 10, // Optional
    offset: 0, // Optional
  }, // Optional
  {
    tableName: "chats", // Optional; `chats` by default
    tools: customToolsList, // Optional
  },
);

Update Chat

Allows to modify chat details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.

By Condition

Updates chat records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.

await rag.chat.update(
  "chats",
  // Updated values
  {
    name: "<NEW_CHAT_NAME>", // Optional
    systemRole: "<NEW_CHAT_SYSTEM_ROLE>", // Optional
    sessionsTableName: "<NEW_SESSIONS_TABLE_NAME>", // Optional
    messagesTableName: "<NEW_MESSAGES_TABLE_NAME>", // Optional
    createdAt: "NEW_CREATED_AT", // Optional
  },
  // Where condition
  {
    id: "<CHAT_ID>", // Optional
    name: "<CHAT_NAME>", // Optional
    systemRole: "<SYSTEM_ROLE>", // Optional
    sessionsTableName: "<SESSIONS_TABLE_NAME>", // Optional
    messagesTableName: "<MESSAGES_TABLE_NAME>", // Optional
  },
);

Specific

Updates a specific chat instance by directly providing the new values for the chat's fields. Each field is optional and only provided fields will be updated.

await chat.update({
  name: "<NEW_CHAT_NAME>", // Optional
  systemRole: "<NEW_CHAT_SYSTEM_ROLE>", // Optional
  sessionsTableName: "<NEW_SESSIONS_TABLE_NAME>", // Optional
  messagesTableName: "<NEW_MESSAGES_TABLE_NAME>", // Optional
  createdAt: "NEW_CREATED_AT", // Optional
});

Delete Chat

Deletes chat records from the database. You can delete chats based on specific conditions or directly target a specific chat instance.

By Condition

Deletes chats, sessions, and messages that match the specified conditions.

await rag.chat.delete(
  "chats",
  "chat_sessions",
  "chat_messages",
  // Where condition
  {
    id: "<CHAT_ID>", // Optional
    name: "<CHAT_NAME>", // Optional
    systemRole: "<SYSTEM_ROLE>", // Optional
    sessionsTableName: "<SESSIONS_TABLE_NAME>", // Optional
    messagesTableName: "<MESSAGES_TABLE_NAME>", // Optional
  },
);

Specific

Deletes a specific chat instance, including its associated sessions and messages.

await chat.delete();

Chat Session

Create Chat Session

Initializes a new chat session. You can optionally provide a name for the session, or create a session with default parameters.

const session = await chat.session.create(
  {
    name: "<CHAT_SESSION_NAME>", // Optional
  }, // Optional
);

Find Chat Session

Retrieves session records, either all available sessions or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.

All Chat Sessions

Fetches all session records without any filtering.

const sessions = await chat.session.find();

By Condition

Fetches a specific session record based on a provided condition, such as id.

const session = await chat.session.find({ where: { id: "<ID>" } });

With Advanced Filtering

Fetches session records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset).

const sessions = await chat.session.find(
  {
    where: { columnName: "COLUMN_VALUE" }, // Optional
    orderBy: { columnName: "asc" }, // Optional
    limit: 10, // Optional
    offset: 0, // Optional
  }, // Optional
);

Update Chat Session

Allows to modify chat session details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.

By Condition

Updates chat session records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.

await chat.session.update(
  // Updated values
  {
    chatID: "<NEW_CHAT_ID>", // Optional
    name: "<NEW_SESSION_NAME>", // Optional
    createdAt: "<NEW_CREATED_AT>", // Optional
  },
  // Where condition
  {
    id: "<ID>", // Optional
    name: "<SESSION_NAME>", // Optional
    createdAt: "<SESSION_CREATED_AT>", // Optional
  },
);

Specific

Updates a specific chat session instance.

await session.update({
  chatID: "<NEW_CHAT_ID>", // Optional
  name: "<NEW_CHAT_NAME>", // Optional
  createdAt: "<NEW_CREATED_AT>", // Optional
});

Delete Chat Session

Deletes chat session records from the database. You can delete chat sessions based on specific conditions or directly target a specific chat session instance.

By Condition

Deletes chat sessions, and messages that match the specified conditions.

await chat.session.delete({
  // Where condition
  id: "<CHAT_SESSION_ID>",
  name: "<CHAT_SESSION_NAME>",
  createdAt: "<CREATED_AT>",
});

Specific

Deletes a specific chat instance, including its associated messages.

await session.delete();

Submit

This method is used to initiate a chat session with a given prompt, model, and various configuration options. You can specify settings such as the model to be used, the history of the chat, database schema loading, and custom tools for function calling. Additionally, it supports streaming responses and handling tool calls and their results

const customToolsList = [describeDatabaseChatTool(database)];

const stream = await session.submit({
  model: "gpt-4o-mini", // Optional; specifies the LLM model to use
  prompt: "<PROMPT>", // The prompt to be sent to the LLM
  systemRole: "<SYSTEM_ROLE>", // Optional; defines the role of the system (e.g., instructions or persona)
  temperature: 0, // Optional; controls the randomness of the output (0 = deterministic, 1 = more creative)
  stream: true, // Optional; enables streaming of responses; `false` by default
  loadHistory: true, // Optional; loads the chat history into the session; `false` by default
  loadDatabaseSchema: true, // Optional; loads the database schema into the session; `false` by default
  messages: [], // Optional; array of messages to preload into the session; `[]` by default
  maxMessagesLength: 2048, // Optional; limits the total length of messages; `2048` by default
  tools: customToolsList, // Optional; list of tools to be used by the LLM during the session; `[]` by default
  onMessagesLengthSlice: () => {}, // Optional; callback triggered when messages length is sliced
  onMessageLengthExceededError: (error) => {}, // Optional; callback for when message length exceeds the limit
  onMessagesLengthExceededError: (error) => {}, // Optional; callback for when messages length exceed the limit
  toolCallHandlers: { toolName: async (tool, toolParams) => {} }, // Optional; handles calls to custom tools
  toolCallResultHandlers: { toolName: async (tool, toolResult, toolParams) => {} }, // Optional; handles the result of tool calls
});

const chatCompletion = await ai.chatCompletions.handleStream(stream, async (chunk) => {
  console.log(chunk); // Process each chunk of the streaming response
});

Chat Message

Create Chat Message

const message = await session.message.create({
  role: "user", // Supported values: `user` | `assistant` | `system`
  content: "<CONTENT>",
});

Find Chat Message

Retrieves chat message records, either all available messages or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.

All Chat Messages

Fetches all chat message records without any filtering.

const messages = await chat.message.find();

By Condition

Fetches a specific message record based on a provided condition, such as id.

const message = await chat.message.find({ where: { id: "<ID>" } });

With Advanced Filtering

Fetches chat message records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset).

const messages = await chat.message.find(
  {
    where: { columnName: "COLUMN_VALUE" }, // Optional
    orderBy: { columnName: "asc" }, // Optional
    limit: 10, // Optional
    offset: 0, // Optional
  }, // Optional
);

Update Chat Message

Allows to modify chat message details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.

By Condition

Updates chat message records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.

await session.message.update(
  // Updated values
  {
    sessionID: "<NEW_SESSION_ID>", // Optional
    role: "<NEW_ROLE>", // Optional; Supported values: `user` | `assistant` | `system`
    content: "<NEW_CONTENT>", // Optional
    createdAt: "<NEW_CREATED_AT>", // Optional
  },
  // Where condition
  {
    id: "<ID>", // Optional
    role: "<MESSAGE_ROLE>", // Optional
    content: "<MESSAGE_CONTENT>", // Optional
    createdAt: "<MESSAGE_CREATED_AT>", // Optional
  },
);

Specific

Updates a specific chat message instance.

await message.update({
  sessionID: "<NEW_SESSION_ID>", // Optional
  role: "<NEW_ROLE>", // Optional; Supported values: `user` | `assistant` | `system`
  content: "<NEW_CONTENT>", // Optional
  createdAt: "<NEW_CREATED_AT>", // Optional
});

Delete Chat Message

Deletes chat message records from the database. You can delete chat messages based on specific conditions or directly target a specific chat message instance.

By Condition

Deletes chat messages that match the specified conditions.

await session.message.delete(
  // Where condition
  {
    id: "<CHAT_ID>", // Optional
    role: "<CHAT_ROLE>", // Optional
    content: "<CHAT_CONTENT>", // Optional
    createdAt: "<CHAT_CREATED_AT>", // Optional
  },
);

Specific

Deletes a specific chat message instance.

await message.delete();

File

Upload File

Uploads a file buffer, creates embeddings based on the file content, and inserts the resulting embeddings into a specified table for use in vector searches. By default, the method stores embeddings in the vectors table, but you can customize the table name and column names.

const buffer = "<FILE_BUFFER>";

await rag.file.upload({
  buffer,
  name: "<FILE_NAME>",
  ext: "<FILE_EXTENSION>", // Supported values: `txt` | `csv` | `pdf`
  tableParams: {
    name: "<TABLE_NAME>", // Optional; `vectors` by default
    contentColumnName: "<CONTENT_COLUMN_NAME>", // Optional; `content` by default
    vColumnName: "<V_COLUMN_NAME>", // Optional; `v_content` by default
  }, // Optional
  textSplitterOptions: {...}, // Optional; ai.textSplitter.split options
  embeddingParams: {...}, // Optional; ai.embeddings.create params
});

Delete File

Deletes a file's corresponding embeddings from the specified table. By default, it removes entries from the vectors table, but you can customize the table name if needed.

await rag.file.delete(
  "<FILE_NAME>",
  "<TABLE_NAME>", // Optional; `vectors` by default
);

Tools

You can create custom tools to extend the functionality of the LLM when creating or finding chats, or when calling the ChatSession.submit method. These tools can also integrate with function calling, allowing the LLM to execute specific functions during interactions. For detailed information on how to create a tool, refer to this guide.

Additionally, this package includes ready-to-use tools, which you can find here.

Below are examples of how to use tools:

import { ChatCompletionTool } from "@singlestore/ai";
import { describeDatabaseChatTool, textToSQLChatTool, vectorSearchChatTool } from "@singlestore/rag";
import { z } from "zod";

const customTool = new ChatCompletionTool({
  name: "<TOOL_NAME>",
  description: "<TOOL_DESCRIPTION>",
  params: z.object({ paramName: z.string().describe("<PARAM_DESCRIPTION>") }),
  call: async (params) => {
    const value = await anyFnCall(params);
    return { name: "<TOOL_NAME>", params, value: JSON.stringify(value) };
  },
});

const tools = [
  customTool,
  describeDatabaseChatTool(database),
  textToSQLChatTool(database, ai, { model: "gpt-4o-mini" }),
  vectorSearchChatTool(database),
];

const chat = await rag.chat.create({ tools });
// OR
const chats = await rag.chat.find({}, { tools });
// OR
const chatCompletion = await session.submit({ tools });