OctoAI Node Library

The OctoAI Node.js library provides convenient access to the OctoAI API from JavaScript/TypeScript.

Documentation

API reference documentation is available here.

Whisper is not currently available. If you're interested in using speech-to-text models, please contact us at [email protected]

Installation

npm install --save octoai
# or
yarn add @octoai/ask

Usage

import { OctoAIClient, OctoAI } from "@octoai/sdk";

const octoai = new OctoAIClient({
    apiKey: "...", // Defaults to process.env.OCTOAI_TOKEN
});

await octoai.textGen.createChatCompletion({
    messages: [
        {
            role: "role",
        },
    ],
    model: "model",
});

Exception Handling

When the API returns a non-success status code (4xx or 5xx response), a subclass of OctoAIError will be thrown:

import { OctoAIError } from '@octoai/sdk';

try {
  await octoai.textGen.createChatCompletion(...);
} catch (err) {
  if (err instanceof OctoAIError) {
    console.log(err.statusCode);
    console.log(err.message);
    console.log(err.body);
  }
}

Retries

The OctoAI Node SDK is instrumented with automatic retries with exponential backoff. A request will be retried as long as the request is deemed retriable and the number of retry attempts has not grown larger than the configured retry limit (default: 2).

A request is deemed retriable when any of the following HTTP status codes is returned:

• 408 (Timeout)
• 409 (Conflict)
• 429 (Too Many Requests)
• 5XX (Internal Server Errors)

Use the maxRetries request option to configure this behavior.

const response = octoai.assetLibrary.list(..., {
  maxRetries: 0 // override maxRetries at the request level
});

Timeouts

The SDK defaults to a 60 second timeout. Use the timeoutInSeconds option to configure this behavior.

const response = octoai.assetLibrary.list(..., {
  timeoutInSeconds: 30 // override timeout to 30s
});

Runtime compatiblity

The SDK defaults to node-fetch but will use the global fetch client if present. The SDK works in the following runtimes:

The following runtimes are supported:

• Node.js 15+
• Vercel
• Cloudflare Workers
• Deno v1.25+
• Bun 1.0+

Customizing Fetch client

The SDK provides a way for you to customize the underlying HTTP client / Fetch function. If you're running in an unsupported environment, this provides a way for you to break the glass and ensure the SDK works.

import { OctoAIClient } from '@octoai/sdk';

const octoai = new OctoAIClient({
  apiKey: "...",
  fetcher: // provide your implementation here
});

Beta status

This SDK is in beta, and there may be breaking changes between versions without a major version update. Therefore, we recommend pinning the package version to a specific version in your package.json file. This way, you can install the same version each time without breaking changes unless you are intentionally looking for the latest version.

Contributing

While we value open-source contributions to this SDK, this library is generated programmatically. Additions made directly to this library would have to be moved over to our generation code, otherwise they would be overwritten upon the next generated release. Feel free to open a PR as a proof of concept, but know that we will not be able to merge it as-is. We suggest opening an issue first to discuss with us!

On the other hand, contributions to the README are always very welcome!