colelawrence-dexaai-dexter-fork
v1.2.8
Published
LLM tools used in production at Dexa, with a focus on real-world RAG.
Downloads
3
Readme
Dexter
Dexter is a set of mature LLM tools used in production at Dexa, with a focus on real-world RAG (Retrieval Augmented Generation).
If you're a TypeScript AI engineer, check it out! 😊
Features
- production-quality RAG
- extremely fast and minimal
- handles caching, throttling, and batching for ingesting large datasets
- optional hybrid search w/ SPLADE embeddings
- minimal TS package w/ full typing
- uses
fetch
everywhere - supports Node.js 18+, Deno, Cloudflare Workers, Vercel edge functions, etc
- full docs
Install
npm install @dexaai/dexter
This package requires node >= 18
or an environment with fetch
support.
This package exports ESM. If your project uses CommonJS, consider switching to ESM or use the dynamic import()
function.
Usage
This is a basic example using OpenAI's text-embedding-ada-002 embedding model and a Pinecone datastore to index and query a set of documents.
import 'dotenv/config';
import { EmbeddingModel } from '@dexaai/dexter/model';
import { PineconeDatastore } from '@dexaai/dexter/datastore/pinecone';
async function example() {
const embeddingModel = new EmbeddingModel({
params: { model: 'text-embedding-ada-002' },
});
const store = new PineconeDatastore({
contentKey: 'content',
embeddingModel,
});
await store.upsert([
{ id: '1', metadata: { content: 'cat' } },
{ id: '2', metadata: { content: 'dog' } },
{ id: '3', metadata: { content: 'whale' } },
{ id: '4', metadata: { content: 'shark' } },
{ id: '5', metadata: { content: 'computer' } },
{ id: '6', metadata: { content: 'laptop' } },
{ id: '7', metadata: { content: 'phone' } },
{ id: '8', metadata: { content: 'tablet' } },
]);
const result = await store.query({ query: 'dolphin' });
console.log(result);
}
Docs
See the docs for a full usage guide and API reference.
Examples
To run the included examples, clone this repo, run pnpm install
, set up your .env
file, and then run an example file using tsx
.
Environment variables required to run the examples:
OPENAI_API_KEY
- OpenAI API keyPINECONE_API_KEY
- Pinecone API keyPINECONE_BASE_URL
- Pinecone index's base URL- You should be able to use a free-tier "starter" index for most of the examples, but you'll need to upgrade to a paid index to run the any of the hybrid search examples
- Note that Pinecone's free starter index doesn't support namespaces,
deleteAll
, or hybrid search :sigh:
SPLADE_SERVICE_URL
- optional; only used for the chatbot hybrid search example
Basic
npx tsx examples/basic.ts
Caching
npx tsx examples/caching.ts
Redis Caching
This example requires a valid REDIS_URL
env var.
npx tsx examples/caching-redis.ts
AI Function
This example shows how to use createAIFunction
to handle function
and tool_calls
with the OpenAI chat completions API and Zod.
npx tsx examples/ai-function.ts
AI Runner
This example shows how to use createAIRunner
to easily invoke a chain of OpenAI chat completion calls, resolving tool / function calls, retrying when necessary, and optionally validating the resulting output via Zod.
Note that createAIRunner
takes in a functions
array of AIFunction
objects created by createAIFunction
, as the two utility functions are meant to used together.
npx tsx examples/ai-runner.ts
Chatbot
This is a more involved example of a chatbot using RAG. It indexes 100 transcript chunks from the Huberman Lab Podcast into a hybrid Pinecone datastore using OpenAI ada-002 embeddings for the dense vectors and a HuggingFace SPLADE model for the sparse embeddings.
You'll need the following environment variables to run this example:
OPENAI_API_KEY
PINECONE_API_KEY
PINECONE_BASE_URL
- Note: Pinecone's free starter indexes don't seem to support namespaces or hybrid search, so unfortunately you'll need to upgrade to a paid plan to run this example. See Pinecone's hybrid docs for details on setting up a hybrid index, and make sure it is using the
dotproduct
metric.
- Note: Pinecone's free starter indexes don't seem to support namespaces or hybrid search, so unfortunately you'll need to upgrade to a paid plan to run this example. See Pinecone's hybrid docs for details on setting up a hybrid index, and make sure it is using the
SPLADE_SERVICE_URL
npx tsx examples/chatbot/ingest.ts
npx tsx examples/chatbot/cli.ts
License
MIT © Dexa