Universal LLM Completion Module Documentation

A powerful TypeScript library for unified access to OpenAI (GPT) and Anthropic (Claude) language models. Perfect for applications requiring robust LLM integration with streaming support and error handling.

Table of Contents

• Features
• Installation
• Quick Start
• Detailed Usage Guide
• Error Handling
• Advanced Features
• API Reference
• TypeScript Support
• Limitations
• Contributing
• License

Features

• ✨ Unified API for GPT and Claude models
• 🔄 Streaming and non-streaming responses
• 🛠️ Function/tool calling with type safety
• 🌐 SSE and WebSocket support
• ⚡ Built-in request cancellation
• 🎯 Comprehensive error handling
• 📊 Token usage tracking
• 💾 Message caching (Claude)
• 🎥 Image generation (OpenAI)
• 📝 Full TypeScript support
• 💾 Message caching with both models

Installation

npm install universal-llm-completion

Quick Start

import { llmCompletion } from 'universal-llm-completion';

const response = await llmCompletion({
  payload: {
    model: "gpt-4",
    messages: [{ role: "user", content: "Hello!" }],
    stream: true
  },
  apiKey: process.env.API_KEY,
  uuid: "unique-id",
  isSocket: false,
  clientObj: res
});

Detailed Usage Guide

Server Implementation

Express (SSE)

app.post('/chat', async (req, res) => {
  try {
    await llmCompletion({
      payload: req.body,
      clientObj: res,
      uuid: req.body.uuid,
      isSocket: false,
      apiKey: process.env.API_KEY
    });
  } catch (error) {
    handleError(error, res);
  }
});

Socket.io

io.on('connection', (socket) => {
  socket.on('chat', async (data) => {
    try {
      await llmCompletion({
        payload: data,
        clientObj: socket,
        uuid: data.uuid,
        isSocket: true,
        apiKey: process.env.API_KEY
      });
    } catch (error) {
      socket.emit('error', formatError(error));
    }
  });
});

Error Handling

Error Response Formats

OpenAI Errors

{
  "error": {
    "message": "Error description",
    "type": "error_type",
    "code": "ERROR_CODE",
    "statusCode": 400
  }
}

try{
  await llmCompletion({
    payload: req.body,
    clientObj: res,
    uuid: req.body.uuid,
    isSocket: false,
    apiKey: process.env.API_KEY
  });
}catch(error){
  if(error instanceof LLMError){
    res.status(error.statusCode).json({message: error.message, type: error.type, code: error.code, statusCode: error.statusCode})
  } else {
    res.status(500).json({message: "Internal server error", statusCode: 500})
  }
}

Common OpenAI Error Codes:

• INVALID_API_KEY (401): Invalid authentication
• RATE_LIMIT_EXCEEDED (429): Rate limit reached
• OPENAI_SERVICE_ERROR (500): Internal server error
• FORBIDDEN_COUNTRY (403): Geographic restriction
• OPENAI_TIMEOUT_ERROR (408): Request timeout
• OPENAI_CONNECTION_ERROR (0): Connection failed

Claude Errors

{
  "error": {
    "message": "Error description",
    "type": "error_type",
    "code": "ERROR_CODE",
    "statusCode": 400
  }
}

Common Claude Error Codes:

• INVALID_API_KEY (401): Authentication failed
• FORBIDDEN_ACCESS (403): Access denied
• CLAUDE_SERVICE_ERR (503): Service unavailable
• REQUEST_TOO_LARGE_ERR (413): Input too large
• CLAUDE_OVERLOAD_ERR (529): Service overloaded
• CLAUDE_TIMEOUT_ERROR (408): Request timeout

Error Status Codes Overview

| Status Code | Description | Common Scenarios | |-------------|-------------|-----------------| | 400 | Bad Request | Invalid input parameters | | 401 | Unauthorized | Invalid API key | | 403 | Forbidden | Geographic restrictions, permissions | | 404 | Not Found | Invalid endpoint or resource | | 408 | Timeout | Request timeout | | 413 | Payload Too Large | Input size exceeded | | 429 | Too Many Requests | Rate limit exceeded | | 500 | Internal Error | Server-side issues | | 503 | Service Unavailable | Temporary outage | | 529 | Service Overload | System capacity reached |

Advanced Features

Function/Tool Calling

const payload = {
  model: "gpt-4",
  messages: [/* your messages */],
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Get weather data",
      parameters: {
        type: "object",
        properties: {
          location: { type: "string" }
        }
      }
    }
  }]
};

Request Cancellation

// Cancel specific request
const cancelled = cancelConnection("request-uuid");

// Cancel all requests
cancelAllConnections();

Client-Side Implementation

SSE Client

const chat = new EventSource('/chat');

chat.onmessage = (event) => {
  const data = JSON.parse(event.data);
  switch(data.type) {
    case 'text':
      console.log('Message:', data.content);
      break;
    case 'llm-tool-call-status':
      console.log('Tool:', data.name, data.status);
      break;
  }
};

WebSocket Client

socket.on('llm-response', (data) => {
  const response = JSON.parse(data);
  updateUI(response.content);
});

socket.on('llm-tool-call-status', (status) => {
  handleToolCall(status);
});

Cache Control

The library supports fine-grained caching control for different content types:

// Message with cache control
const message = {
  role: "user",
  content: [{
    type: "text",
    text: "Hello",
    cache_control: {
      type: "ephemeral" // or other cache type
    }
  }]
};

// Tool with cache control
const tool = {
  type: "function",
  function: {
    name: "get_weather",
    description: "Get weather data",
    parameters: {/* ... */},
    cache_control: {
      type: "ephemeral"
    }
  }
};

Cache control can be applied to:

• Individual message content blocks (text/images)
• System messages
• Function/tool definitions
• Image content

The library handles cache control differently for OpenAI and Claude:

• For OpenAI: requests are automatically cached
• For Claude: Cache control properties are preserved and passed through

Cache Control Types

• ephemeral: Content not stored permanently (offered by Claude, GPT autmatically caches)
• Custom types can be defined as strings

API Reference

Main Functions

llmCompletion

function llmCompletion(options: LLMCompletionRequest): Promise<LLMCompletionResponse>

cancelConnection

function cancelConnection(uuid: string): boolean

Type Definitions

interface LLMCompletionRequest {
  payload: LLMPayload;
  clientObj?: Response | Socket;
  uuid: string;
  isSocket: boolean;
  apiKey: string;
}

interface LLMCompletionResponse {
  completion: string;
  toolCalls: (OpenAiToolCall | ClaudeToolCall)[];
  usage: Usage;
  cancelled: boolean;
}

//ToolCall can be of type OpenAiToolCall or ClaudeTool
export type OpenAiToolCall = {
  index: number
  id: string
  type: 'function' | 'tool'
  function: {
    name: string
    arguments: string
  }
  status: 'incomplete' | 'complete'
}

export type ClaudeToolCall = {
  name: string | undefined
  id: string | undefined
  type?: 'tool' | 'function'
  input: string | undefined
  status?: 'incomplete' | 'complete'
  partialJson?: string
}

Environment Setup

# Required environment variables
OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key

# Optional configurations
MAX_TOKENS=2000
DEFAULT_TEMPERATURE=0.7

Limitations

• No support for image generation
• Audio processing not available

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

License

MIT License - see LICENSE for details.