@dingyi222666/js-tiktoken
v1.0.15
Published
JavaScript port of tiktoken
Downloads
487
Readme
⏳ js-tiktoken
tiktoken is a BPE tokeniser for use with OpenAI's models. This is a pure JS port of the original tiktoken library.
Install the library from NPM:
npm install @dingyi222666/js-tiktoken
This fork remove the default encoding files, you need to load the encoding files by yourself.