llm-load-balancer
v0.3.1
Published
A small service package allowing to perform load balancing of LLM prompts between models, regions and model hosting platforms
Downloads
963
Maintainers
Readme
llm-load-balancer
A small npm package enabling load balancing LLM prompts between models, regions, hosting platforms and even individual model configurations.