LLM Agents

A library to create LLM Agents with Node.js

Usage

$ bun install llm-agents

# or the legacy way

$ npm install llm-agents

Actions

Extends the LLMAction class to define an action with:

• name
• usage
• parameters

import { execSync } from 'child_process';

import { Action, ActionFeedback } from 'llm-agents';

type ExecuteShellCommandActionParametersNames = 'command';

export class ExecuteShellCommandAction extends LLMAction<ExecuteShellCommandActionParametersNames> {
  public name = 'executeShellCommand';
  public usage = 'execute a shell command';
  public parameters = [
    {
      name: 'command' as const,
      usage: 'command to execute',
    },
  ];

  protected async executeAction(
    parameters: Record<ExecuteShellCommandActionParametersNames, string>
  ): Promise<ActionFeedback> {
    const { command } = parameters;

    try {
      const result = execSync(command);

      return {
        message: `$ ${command}\n${result.toString()}`,
        type: 'success',
      };
    } catch (error) {
      return {
        message: `$ ${command}\n${error.message}`,
        type: 'error',
      };
    }
  }
}

Examples:

• ListFilesActions
• CopyFileActions
• CreateDirectoryActions

Agents

Extends the LLMAgent class to define an Agent with:

• template content
• template formating
• actions (optionnal)

import { PromptTemplate } from 'langchain/prompts';

import { LLMAction, FileCache, LLMAgent } from 'llm-agents';

import {
  CreateDirectoryAction,
  CopyFileAction,
  ListFilesAction,
} from './actions';

export class BackupAgent extends LLMAgent {
  private source: string;
  private destination: string;

  protected template = new PromptTemplate({
    template: `You are a backup agent capable of moving files from one place to another.

You task is to backup the files containing the typescript code of a project.
You need to copy the files containing the code into a backup directory.
You can only move one file after another.

You can use the following actions:
# BEGIN ACTIONS DEFINITION
{actions}
# END ACTIONS DEFINITION

The last action result was:
# BEGIN LAST ACTION RESULT
{feedback}
# END LAST ACTION RESULT

Ensure you are copying every files in every directories from the source.
The source directory is {source} and the destination directory is {destination}.
Skip node_modules directories.

Start by a sentence summarizing the current state of your task according to the last action result.
Then, answer with the actions you want to execute.
`,
    inputVariables: ['source', 'destination', 'actions', 'feedback'],
  });

  constructor({
    source,
    destination,
  }: {
    source: string;
    destination: string;
  }) {
    super({
      actions: [
        new ListFilesAction(),
        new CopyFileAction(),
        new CreateDirectoryAction(),
      ],
      cacheEngine: new FileCache(),
    });

    this.source = source;
    this.destination = destination;
  }

  protected async formatPrompt({
    actions,
    feedbackSteps,
  }: {
    actions: string;
    feedbackSteps: string[];
  }) {
    return this.template.format({
      source: this.source,
      destination: this.destination,
      actions,
      feedback: feedbackSteps.join('\n\n'),
    });
  }
}

Examples:

• BackupAgent

Cache

During development phase, it's advised to use the filesystem cache.

It will save both prompt and answer for each step.

The cache key is a hash of the prompt so if the prompt does not change, the cached answer will be used directly.

You can also debug your prompts and answers in the .cache/ folder. In development mode, they will be prefixed by the step number (e.g. 0-92957a2b27-answer.txt)

Tests

Unit tests can be run with Bun: bun test

See tests/unit

Integration tests consist in a benchmark of LLMs agents:

• BackupAgent: bun tests/prompt-engineering/backup-agent/run-backup-agent-test.ts