Usage

npm install tip20

Bun is still the preferred runtime (bun install tip20).

tip20 includes both a streaming and a regular async function for output.

shortCircuit is a boolean that will stop the streaming once enough is found in the response. This should reduce response times by about 90%.

import fs from "fs";
import { tip20streaming } from "tip20";

const tip20stream = await tip20streaming(
  "typescript",
  "Some mixed response",
  "claude-3-haiku-20240307",
  true // short ciruit.
);

for await (const token of tip20stream) {
  console.log("Got packet ", token);
}

The tests folder has more examples.

Tip20 is named after the (half serious) joke that tipping LLMs results in better performance. Using prefix completion (on supported LLMs - currently for the package that's Haiku and Sonnet) you can get streaming results of extracted data from almost any input.

from @voooooogel

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. TODOs.md is a good place to start.