💫StarCoder.js *Experimental*

THIS IS UNDER DEVELOPMENT AND IS NOT PRODUCTION READY. BROWSER REQUIRES MEM64 SUPPORT TO RUN THIS PROJECT

This project brings starcoder.cpp to browser with power of WebAssembly.

Demo

Web Application : starcoder.js

Performance

Following performance speed is captured on Firefox Nightly with following system configuration:

• CPU: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
• Memory: 24GB

Build Instructions

git clone https://github.com/bigcode-project/starcoder.cpp
cd starcoder.cpp

Model Quantization

# make sure to have torch, transformer library
python convert-hf-to-ggml.py bigcode/tiny_starcoder_py 

# Build local
make clean
make 

# Taken from here https://huggingface.co/mike-ravkine/tiny_starcoder_py-GGML/blob/main/tiny_starcoder_py-fp16.bin
./quantize models/tiny_starcoder_py-fp16.bin models/tiny_starcoder_py-q4_1.bin 3

# Build WASM/JS

## Step 1: Build for WASM
make clean
make CC=emcc CXX=em++ LLAMA_NO_ACCELERATE=1 CFLAGS=" -DNDEBUG -s MEMORY64" CXXFLAGS=" -DNDEBUG -s MEMORY64" LDFLAGS="-s MEMORY64 -s TOTAL_MEMORY=2147483648 -s STACK_SIZE=524288  --preload-file models" main.html

or

./build-wasm.sh

## Step 2: Manually patch docs/main.js

from:
var REMOTE_PACKAGE_BASE = 'main.data';

to:
var REMOTE_PACKAGE_BASE = 'https://media.githubusercontent.com/media/rahuldshetty/starcoder.js/main/docs/main.data';