@cloudflare/ai-utils
v1.0.1
Published
Utilities for working with AI tools
Downloads
7,037
Maintainers
Keywords
Readme
Utilities to make it easier to work with Workers AI platform.
Installation
npm install @cloudflare/ai-utils --save
What is this?
This package contains a set of utilities to make it easier to work with the Cloudflare Workers AI platform.
To learn more about function calling and its usage on Cloudflare, refer to the documentation here: Cloudflare Function Calling Documentation.
This package contains the following utilities:
runWithTools
runWithTools
is a helper function that helps you do function calling with a simpler syntax with AI.run()
. You can provide a function inline with the tool you want to use, which is run automatically with the arguments passed to the function in the same worker where the AI call is invoked.
Features of runWithTools:
- ⚡ Multi-tool calls feature
- 🔁 Recursive tool calls
- 👟 Handles function runs with errors
- Streaming final response
- The returntype is the same as AI.run
- Programmatically trim tools, or use
autoTrimTools
to automatically do it.
It also supports the following configuration options:
strictValidation
- If set totrue
, the function will throw an error if the tool's arguments by LLM are not valid. Defaults tofalse
.maxRecursiveToolRuns
- The maximum number of times the tool can be run recursively. Defaults to1
.streamFinalResponse
- If set totrue
, the function will return aReadableStream
instead of a string. Defaults tofalse
.verbose
- If set totrue
, the function will print out more information about the tool's execution. Defaults tofalse
.trimFunction
- This is a function that can be used to trim the tools. It takes all the arguments (tools, AI, model, messages) and can return the trimmed array of tools. You can use this to decide which tools to use with a small model before sending the function call query (which usually uses a lot of tokens!)
We also provide a function autoTrimTools
, which takes a tool array and trims it based on the names and the descriptions of each of them.
Usage:
const r = await runWithTools(
env.AI,
"@hf/nousresearch/hermes-2-pro-mistral-7b",
{
messages: [{ role: "user", content: "What's the weather in Mumbai, India?" }],
tools: [
{
name: "get-weather",
description: "Gets weather information of a particular city",
parameters: {
type: "object",
properties: {
city: {
type: "string",
description: "The city name"
}
},
required: ["city"]
},
function: async ({ city }) => {
// fetch weather data from an API
console.log("value from llm", city)
return city
}
}
]
},
{
strictValidation: true,
maxRecursiveToolRuns: 1,
streamFinalResponse: true,
// If there's too many tools, you can enable this
trimFunction: autoTrimTools
}
)
You may also use the tool()
function, which is a helper that provides type completions for the function arguments and the tool, so you can create tools from outside the runWithTools
function and still enjoy type safety.
import { tool } from "@cloudflare/ai-utils"
const ourTool = tool({
name: "ourTool"
...
function: async (args) => {...}
})
OpenAPI to Tool Converter
Function createToolsFromOpenAPISpec
This project provides a function createToolsFromOpenAPISpec
that converts an OpenAPI specification into a set of tools in JSONSchema format, each with an associated function to make API calls. This can be useful for generating API clients dynamically based on an OpenAPI spec.
Using this, users can simply make agents like:
- Google calendar agent
- Github repository manager / issue tracker
- Email -> Airtable database inserter
As long as the API provides an OpenAPI spec, an agent can be made with it. When paired with runWithTools, this provides an amazing developer experience where smart workloads can be performed with just a few lines of code.
Usage
This function takes an OpenAPI spec and a configuration object and returns an array of tools.
Arguments;
spec
: The OpenAPI specification. This can be in JSON or YAML form. You can also provide a remote specification and it will be fetched and parsed.config
:matchPatterns
: An array of regex patterns to match, helpful if you don't want the entire API but a small part of it.overrides
[]:matcher
: A matcher function that takes the URL and method and returns a boolean.values
: If the matcher was true, these values will be added as an override when making the actual fetch request. This can be helpful for adding User agents, authentication tokens and more. You can provideheaders
,pathData
,query
,formData
,body
andcookies
as values.
const githubUserTool = await createToolsFromOpenAPISpec(GITHUB_SPEC, {
matchPatterns: [
// api.github.com/users/{username} and api.github.com/users/{username}/repos
/^https:\/\/api\.github\.com\/users\/([^\/]+)\/repos$/,
/^https:\/\/api\.github\.com\/users\/([^\/]+)$/,
// Also, for api.github.com/repos/{owner}/{repo}/ queries
/^https:\/\/api\.github\.com\/repos\/([^\/]+)\/([^\/]+)\/?$/
],
overrides: [
{
// for all requests on *.github.com, we'll need to add a User-Agent and Authorization.
matcher: ({ url, method }) => {
return url.hostname === "api.github.com"
},
values: {
headers: {
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
Authorization: "Bearer GITHUB_TOKEN"
}
}
}
]
})
Embedded function calling on the Workers platform
When used with the Cloudflare Workers platform, you can also use bindings to make database queries and lookups, store and get images, spin up browser instances or even run AI models as a function of your tool.
Because we use bindings, most of the compute and the lookups happen in the same execution environment, which helps decrease the latency and get a 10-20% faster response time.
Here's an example of an agent that automatically makes database entries with conversations to have a 'memory' of who the user is:
const r = await runWithTools(env.AI, "@hf/nousresearch/hermes-2-pro-mistral-7b", {
messages: [
{
role: "system",
content: "Save important information about the user to the KV databse."
},
{
role: "user",
content: "Hi! I'm dhravya. An engineering intern at Cloudflare."
}
],
tools: [
tool({
name: "Save information",
description: "Info save about the user",
parameters: {
type: "object",
properties: {
key: {
type: "string",
description: "Name of the user"
},
value: {
type: "string",
description: "Information of the user"
}
},
required: ["key", "value"]
},
function: async ({ key, value }) => {
await env.KV.put(key, value)
return `Saved ${key} to the KV database.`
}
})
]
})
Contributing
This project is open source and welcomes contributions. If you'd like to contribute, please fork the repository and submit a pull request.
License
This project is licensed under the Apache 2.0 License - see the LICENSE file for details.