llm
v1.0.7
Published
Manipulate any language model from the command line.
Downloads
24
Readme
llm: language models usage made easy
Manipulate any language model from the command line.
From simple to advanced usage.
$ llm "Hello world."
Hello there! How can I assist you today?
Leave a ⭐ star to support the project.
Models
Some models are still being added. This is a work in progress.
| Model Name | Status | Description | |------------------------------|--------|-------------------------------------------------| | EVERY OpenAI model | ✅ | | | gpt-3.5-turbo | ✅ | ChatGPT | | gpt-4 | ✅ | GPT-4 via API (waitlist) | | text-davinci-003 | ✅ | InstructGPT (GPT-3) | | llama2 | ✅ | Meta's Llama 2 | | bing-chat | ✅ | Bing Chat: creative, balanced, precise | | bert | ✅ | BERT by Google | | llama-7b-hf | ✅ | Famous llama model | | wizardlm-13b-uncensored | ✅ | WizardLM 30B | | guanaco-65b-gptq | ✅ | Guanaco 65B | | bard | 🔄 | Google Bard | | ... HuggingFace 🤗 models | ✅ | every text-generation model |
Other models can be installed using the --install
command.
Features
| Feature | Status |Comment | |-------------------------|-----------------|---| | Prompt | ✅ |Prompt model with default parameters| | Parameterization | ✅ |temperature, max-length, top-p, top-k, ...| | ChatGPT Plugins | 🔄 | Use chatGPT plugins. web-pilot working, global plugin system in development| | Use files | ✅ |Query models using prompt files| | Prompt chaining | ✅ |Call prompts like functions| | Prompt templating | 🔄 |Use variables in prompt files |
Getting started
git clone https://github.com/snwfdhmp/llm && cd llm
yarn install
make an alias llm
alias llm="node $(pwd)/main.js"
add it to your
.bashrc
or.zshrc
to make it permanent.
You're ready to go ! Try:
$ llm "Hello world"
$ llm -m bing-creative "Tell me a joke"
$ llm -m gpt-3.5-turbo "Tell me a joke"
Usage
Simple prompting with defaults parameters
$ llm "what is the meaning of life?"
Use a specific model
$ llm -m bing-creative "find project ideas to learn react"
Use custom parameters
$ llm --max-length 512 --temperature 1 --top-p 0.9 --top-k 60 "follow the instructions."
List available models
$ llm ls
Name LastUsedAt Author Description
text-davinci-003 2021-10-10 OpenAI InstructGPT by OpenAI
gpt-3.5-turbo 2021-10-10 OpenAI ChatGPT by OpenAI
gpt-4-web 2021-10-10 OpenAI GPT-4 by OpenAI via chatGPT
llama 2021-10-10 Meta Meta's Llama
bard 2021-10-10 Google Google Bard
...
Use files as prompts
$ llm -f ./prompt.txt
Incoming:
- Conversation system (remember past messages)
- Install 3rd party models
- Chaining
$ llm -s session_name "what is the meaning of life?"
remembers past messages
$ llm --install github.com/snwfhdmp/llm-descriptor-llama
downloads model from github
Add any model
Any model can be plugged into llm
using a model descriptor.
Example of a model descriptor which requires installation
kind: llm/descriptor/v1
metadata:
name: llama
model:
install: |
git clone ...
cd ...
./install.sh
# or
docker pull ...
# or
none
usage:
./model-executor -f model.bin $LLM_PARAM_PROMPT
parameters:
LLM_PARAM_PROMPT:
type: string
description: The prompt to use
default: "Hello world"
LLM_PARAM_MAX_TOKENS:
type: int
description: The maximum length of context
default: 100
LLM_PARAM_TEMPERATURE:
type: float
description: The temperature of the model
default: 0.7
Example of a model descriptor which uses an API
kind: llm/descriptor/v1
metadata:
name: llama
model:
install: |
read -p "Enter your API key:" LLM_API_KEY
echo "LLM_API_KEY=$LLM_API_KEY" >> ~/.bashrc
usage: curl -s $LLM_PARAM_API_TARGET_URL -d "prompt=$LLM_PARAM_PROMPT&api_key=$LLM_API_KEY"
parameters:
LLM_PARAM_API_TARGET_URL:
type: string
description: The URL of the API
default: "https://api.llm.com"
LLM_PARAM_PROMPT:
type: string
description: The prompt to use
default: "Hello world"
LLM_PARAM_MAX_TOKENS:
type: int
description: The maximum length of context
default: 100
LLM_PARAM_TEMPERATURE:
type: float
description: The temperature of the model
default: 0.7
Env variables
These variables can be used to tweak llm
behavior.
LLM_DEFAULT_MODEL
- The default model to use when no model is specifiedLLM_ENABLED_PLUGINS
- A comma-separated list of plugins to enableOPENAI_ORGANIZATION_ID
- The organization ID to use for OpenAI models
Roadmap
Project vision and information can be found in docs.
Contributing
Contribute easily by leaving a ⭐ star to the project.
Code contributions are welcome. Please open an issue or a pull request.
Join the team at discord.gg/ccDghPeAT9.