nodemem-js

A simple, in-memory vector database for Node.js, inspired by LangMem.

Description:

nodemem-js is a lightweight, in-process vector database written in JavaScript for Node.js. It's designed to provide fast similarity search over vector embeddings, making it suitable for use cases like:

• Memory for chat agents: Storing and retrieving conversational context based on semantic similarity.
• Retrieval Augmented Generation (RAG): Quickly finding relevant documents or passages from a knowledge base.
• Semantic search in Node.js applications: Any application needing to search for data based on vector embeddings.

Features:

• In-memory storage: Fast and efficient for smaller to medium-sized datasets.
• Cosine similarity search: Finds vectors most similar to a query vector.
• Persistence: Save and load memory to/from JSON files.
• Simple JavaScript API: Easy to use and integrate into Node.js projects.

Installation:

npm install nodemem-js

Usage:

const VectorMemory = require('nodemem-js');

const memory = new VectorMemory();

// Add vectors to memory
memory.add([0.1, 0.2, 0.7], "Document about topic A", { topic: "A" });
memory.add([0.8, 0.1, 0.1], "Document about topic B", { topic: "B" });
memory.add([0.2, 0.7, 0.1], "Document also about topic A", { topic: "A" });

// Query for similar vectors
const queryVector = [0.15, 0.3, 0.55]; // Vector related to topic A
const results = memory.query(queryVector, 2); // Get top 2 results

console.log("Top 2 similar documents:");
results.forEach(result => {
    console.log("- Text:", result.text);
    console.log("  Metadata:", result.metadata);
});

// Save memory to file
memory.save('my_memory.json');

// Load memory from file
const loadedMemory = new VectorMemory();
loadedMemory.load('my_memory.json');

// Query the loaded memory
const loadedResults = loadedMemory.query(queryVector, 1);
console.log("\nTop result from loaded memory:", loadedResults[0].text);

**API Reference:**

## API Reference

### `class VectorMemory`

Represents an in-memory vector database for storing and querying vector embeddings.

#### `constructor()`

```javascript
constructor()

Limitations (Current):

In-memory only: Data is lost when the Node.js process exits unless saved to a file.

Pure JavaScript implementation: Performance might be limited for very large datasets compared to native implementations (Rust, C++, etc.).

Basic cosine similarity: Currently only supports cosine similarity.

Initializes a new VectorMemory instance with an empty memory store.

Parameters:

None

Returns:

A new VectorMemory object.

Example:

const memory = new VectorMemory();
Use code with caution.
JavaScript
add(vector, text, metadata)
add(vector: number[], text?: string | null, metadata?: object | null): void
Use code with caution.
JavaScript
Adds a new vector embedding and associated data to the memory store.

Parameters:

vector (number[]): An array of numbers representing the vector embedding. Must be an array of finite numbers.

text (string | null, optional): Optional text associated with the vector. Defaults to null.

metadata (object | null, optional): Optional metadata object associated with the vector. Defaults to null. Can be any JavaScript object to store additional information.

Returns:

void - This method does not return a value.

Throws:

Error: If vector is not an array or if it contains non-numeric values.

Example:

memory.add([0.2, 0.8, 0], "Information about cats", { category: "animals", type: "feline" });
memory.add([0.7, 0.1, 0.2], "Details on dogs", { category: "animals", type: "canine" });
Use code with caution.
JavaScript
query(queryVector, k)
query(queryVector: number[], k?: number): object[]

Performs a similarity search for vectors in the memory store that are most similar to the queryVector. Uses cosine similarity to measure similarity.

Parameters:

queryVector (number[]): The vector to query against. Must be an array of finite numbers.

k (number, optional): The number of top similar results to return. Defaults to 1. If k is greater than the number of items in memory, all items will be returned.

Returns:

object[]: An array of the top k most similar memory items. Each item in the array is an object with the following properties:

vector (number[]): The vector embedding.

text (string | null): The associated text (if any).

metadata (object | null): The associated metadata (if any).

Returns an empty array ([]) if the memory store is empty or if no similar vectors are found.

Throws:

Error: If queryVector is not an array or if it contains non-numeric values.

Example:

const queryVector = [0.25, 0.75, 0]; // Vector related to cats
const topResults = memory.query(queryVector, 2); // Get top 2 most similar

topResults.forEach(result => {
    console.log("- Text:", result.text);
    console.log("  Metadata:", result.metadata);
});
Use code with caution.
JavaScript
save(filepath)
save(filepath: string): void
Use code with caution.
JavaScript
Saves the current memory store to a JSON file.

Parameters:

filepath (string): The path to the file where the memory should be saved.

Returns:

void - This method does not return a value.

Throws:

Error: If there is an error during file writing (e.g., permission issues, disk errors).

Example:

memory.save('my_vector_memory.json');
Use code with caution.
JavaScript
load(filepath)
load(filepath: string): void
Use code with caution.
JavaScript
Loads memory from a JSON file into the current memory store. If the file is not found, it will start with an empty memory store and log a warning.

Parameters:

filepath (string): The path to the JSON file from which to load memory.

Returns:

void - This method does not return a value.

Behavior when file is not found:

If the file specified by filepath does not exist, the method will:

Log a warning message to the console indicating that the file was not found.

Initialize the memoryStore to an empty array, effectively starting with empty memory.

Will not throw an error in this case.

Throws:

Error: If there is an error during file reading or JSON parsing (other than "file not found").

Example:

const loadedMemory = new VectorMemory();
loadedMemory.load('my_vector_memory.json');
Use code with caution.
JavaScript
Note: All methods operate synchronously. For asynchronous file operations (if needed for larger datasets or non-blocking I/O), consider future enhancements to use asynchronous file system APIs.

**How to use this in your `README.md`:**

1.  Copy and paste the entire Markdown code block above into your `README.md` file, right after the "Usage" section and before "Limitations".
2.  Make sure the Markdown formatting is rendered correctly when you view your `README.md` file (e.g., on GitHub or in a Markdown previewer). Headings should be properly sized, code blocks should be formatted, etc.

This detailed API reference should provide users with a clear understanding of how to use the `VectorMemory` class and its methods. Let me know if you would like any adjustments or further refinements to this API documentation!

Contributing:(spockstein)

Future Enhancements (Ideas):

More similarity metrics: Add support for other distance/similarity metrics (e.g., dot product, Euclidean distance).

Metadata filtering: Allow filtering query results based on metadata.

More efficient data structures (potential): Explore more optimized in-memory data structures for larger datasets if needed.

Consider native addons/WebAssembly (for performance): If performance becomes a bottleneck, investigate using C++ addons or WebAssembly for performance-critical parts.