stream-ai-sdk

The prototype of the Open AI Streaming Service with a frontend SDK.

What does it do?

A typical API call to Open AI only returns results in around 10 seconds. But when you use ChatGPT you see a response straight away, as it prints it out letter by letter (well, token by token actually).

This package provides a prototype of the end-to-end service/SDK that simplifies streaming.

More specifically

Just call getOpenAiStream(params, onStream) or getOpenAiJsonStream(params, onStream) and it'll automatically stream a response for you. Namely, it'll call the onStream callback when it has something to stream.

For example:

getOpenAiTextStream(
  {
    openAiApiKey: 'YOUR API KEY',
    prompt: "What's the meaning of life?",
  },
  (result) => {
    // do something with result
  },
);

Works with JSON responses as well

Streaming text data is not difficult, but what if your prompt asks for JSON data? Then streaming responses will have a partial malformed JSON and your JSON.parse() will fail. This package handles it for you:

const example = [
  {
    firstName: "John",
    lastName: "Doe",
    age: 25,
    favouriteFood: ["Pizza", "Pasta"],
  },
  {
    firstName: "Mary",
    lastName: "Ann",
    age: 27,
    favouriteFood: ["Sushi", "Ramen"],
  },
];

getOpenAiJsonStream<>(
  {
    openAiApiKey: '[YOUR API KEY]',
    prompt: `Generate a JSON array of 10 users following this example: ${JSON.stringify(example)}`,
  },
  (result) => {
    // do something with result
  }}
);

For JSON data you can also use TypeScript generics:

getOpenAiJsonStream<ExpectedType>(params, callback);

How does it work?

The SDK works together with our cloud service and together they handle all the streaming for you.

Can't you build streaming yourself?

Yes, you can. But you'll have to first build a backend for that and then deal with a streamed data on the frontend. This package removes all the complexity.

This is a prototype - do not use it i production

If you use it in production you will expose your Open API API Key.

Is your Open AI Key safe?

Yes, as long as you don't use this library in production. This library and the underlying cloud service never store your Open AI key.