npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

llm-interface

v2.0.1495

Published

A simple, unified NPM-based interface for interacting with multiple Large Language Model (LLM) APIs, including OpenAI, AI21 Studio, Anthropic, Cloudflare AI, Cohere, Fireworks AI, Google Gemini, Goose AI, Groq, Hugging Face, Mistral AI, Perplexity, Reka A

Downloads

273

Readme

llm-interface

Star on GitHub Fork on GitHub Watch on GitHub

Version 2.0.1495 License: MIT Built with Node.js

Introduction

LLM Interface is an npm module that streamlines your interactions with various Large Language Model (LLM) providers in your Node.js applications. It offers a unified interface, simplifying the process of switching between providers and their models.

The LLM Interface package offers comprehensive support for a wide range of language model providers, encompassing 36 different providers and hundreds of models. This extensive coverage ensures that you have the flexibility to choose the best models suited to your specific needs.

Extensive Support for 36 Providers and Hundreds of Models

LLM Interface supports: AI21 Studio, AiLAYER, AIMLAPI, Anyscale, Anthropic, Cloudflare AI, Cohere, Corcel, DeepInfra, DeepSeek, Fireworks AI, Forefront AI, FriendliAI, Google Gemini, GooseAI, Groq, Hugging Face Inference, HyperBee AI, Lamini, LLaMA.CPP, Mistral AI, Monster API, Neets.ai, Novita AI, NVIDIA AI, OctoAI, Ollama, OpenAI, Perplexity AI, Reka AI, Replicate, Shuttle AI, TheB.ai, Together AI, Voyage AI, Watsonx AI, Writer, and Zhipu AI.

AI21 Studio AIMLAPI Anthropic Anyscale Cloudflare AI Cohere Corcel DeepInfra DeepSeek Forefront AI GooseAI Lamini Mistral AI Monster API Neets.ai Perplexity AI Reka AI Replicate Shuttle AI Together AI Writer

Detailed Provider List

Features

  • Unified Interface: LLMInterface.sendMessage is a single, consistent interface to interact with 36 different LLM APIs (34 hosted LLM providers and 2 local LLM providers).
  • Chat Completion, Streaming and Embeddings: Supports chat completion, streaming, and embeddings (with failover).
  • Dynamic Module Loading: Automatically loads and manages LLM interfaces only when they are invoked, minimizing resource usage.
  • Error Handling: Robust error handling mechanisms to ensure reliable API interactions.
  • Extensible: Easily extendable to support additional LLM providers as needed.
  • Response Caching: Efficiently caches LLM responses to reduce costs and enhance performance.
  • Graceful Retries: Automatically retry failed prompts with increasing delays to ensure successful responses.
  • JSON Output: Simple to use native JSON output for various LLM providers including OpenAI, Fireworks AI, Google Gemini, and more.
  • JSON Repair: Detect and repair invalid JSON responses.

Updates

v2.0.14

  • Recovery Mode (Beta): Automatically repair invalid JSON objects in HTTP 400 response errors. Currently, this feature is only available with Groq.

v2.0.11

  • New LLM Providers: Anyscale, Bigmodel, Corcel, Deepseek, Hyperbee AI, Lamini, Neets AI, Novita AI, NVIDIA, Shuttle AI, TheB.AI, and Together AI.
  • Caching: Supports multiple caches: simple-cache, flat-cache, and cache-manager. flat-cache is now an optional package.
  • Logging: Improved logging with the loglevel.
  • Improved Documentation: Improved documentation with new examples, glossary, and provider details. Updated API key details, model alias breakdown, and usage information.
  • More Examples: LangChain.js RAG, Mixture-of-Agents (MoA), and more.
  • Removed Dependency: @anthropic-ai/sdk is no longer required.

Dependencies

The project relies on several npm packages and APIs. Here are the primary dependencies:

  • axios: For making HTTP requests (used for various HTTP AI APIs).
  • @google/generative-ai: SDK for interacting with the Google Gemini API.
  • dotenv: For managing environment variables. Used by test cases.
  • jsonrepair: Used to repair invalid JSON responses.
  • loglevel: A minimal, lightweight logging library with level-based logging and filtering.

The following optional packages can added to extend LLMInterface's caching capabilities:

  • flat-cache: A simple JSON based cache.
  • cache-manager: An extendible cache module that supports various backends including Redis, MongoDB, File System, Memcached, Sqlite, and more.

Installation

To install the LLM Interface npm module, you can use npm:

npm install llm-interface

Quick Start

  • Looking for API Keys? This document provides helpful links.
  • Detailed usage documentation is available here.
  • Various examples are also available to help you get started.
  • A breakdown of model aliases is available here.
  • A breakdown of embeddings model aliases is available here.
  • If you still want more examples, you may wish to review the test cases for further examples.

Usage

First import LLMInterface. You can do this using either the CommonJS require syntax:

const { LLMInterface } = require('llm-interface');

then send your prompt to the LLM provider:

LLMInterface.setApiKey({ openai: process.env.OPENAI_API_KEY });

try {
  const response = await LLMInterface.sendMessage(
    'openai',
    'Explain the importance of low latency LLMs.',
  );
} catch (error) {
  console.error(error);
}

if you prefer, you can pass use a one-liner to pass the provider and API key, essentially skipping the LLMInterface.setApiKey() step.

const response = await LLMInterface.sendMessage(
  ['openai', process.env.OPENAI_API_KEY],
  'Explain the importance of low latency LLMs.',
);

Passing a more complex message object is just as simple. The same rules apply:

const message = {
  model: 'gpt-4o-mini',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain the importance of low latency LLMs.' },
  ],
};

try {
  const response = await LLMInterface.sendMessage('openai', message, {
    max_tokens: 150,
  });
} catch (error) {
  console.error(error);
}

LLMInterfaceSendMessage and LLMInterfaceStreamMessage are still available and will be available until version 3

Running Tests

The project includes tests for each LLM handler. To run the tests, use the following command:

npm test

Current Test Results

Test Suites: 9 skipped, 93 passed, 93 of 102 total
Tests:       86 skipped, 784 passed, 870 total
Snapshots:   0 total
Time:        630.029 s

Note: Currently skipping NVIDIA test cases due to API issues, and Ollama due to performance issues.

TODO

  • [ ] Provider > Models > Azure AI
  • [ ] Provider > Models > Grok
  • [ ] Provider > Models > SiliconFlow
  • [ ] Provider > Embeddings > Nomic
  • [ ] Feature > Image Generation?

Submit your suggestions!

Contribute

Contributions to this project are welcome. Please fork the repository and submit a pull request with your changes or improvements.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Blogs

Share

Twitter Facebook LinkedIn