Integrate Julia and Javascript with Node.js extensions

This repository shows how to build a Node.js native extension that calls Julia code from a precompiled system image. For an introduction in video, see the JuliaCon 2020 talk.

Overview • Requirements • Running the extension • Building the extension • Resources

Overview

In this repository we consider the following use case:

• You have an algorithm written in Julia, and you want to use this algorithm in a Node.js environment.
• You can't or you don't want to install Julia on your servers.
• You want to avoid Julia compilation costs and you want to share data efficiently between the Julia and Node.js runtimes.

To achieve this goal, we propose a two-step approach:

1. Build a Julia system image which includes your precompiled code.
2. Build a Node.js native extension in C/C++ which glues together the Julia and Node.js runtimes.

The main drawbacks of this approach are:

• It requires to write C/C++ glue code.
• Changes to the Julia code requires re-building the system image, and (potentially) adapting the C/C++ code.

We use the following tools:

• PackageCompiler.jl to build the Julia system image.
• node-addon-api a C++ wrapper of N-API to write the Node.js extension.
• node-gyp to build the extension.
• node-pre-gyp to package and distribute the extension.

Repository structure

├── binding.gyp        # Build system configuration
├── example            # Example Express API
│   ├── app.js           # Server
│   └── test.js          # Client
├── index.js           # Extension loader
├── julia              # Julia system image
│   ├── build.jl         # System image build script
│   └── Clustering.jl    # Target package
├── node               # Node.js extension
│   ├── binding.cc       # Extension code
│   ├── extra.h          # Helpers for the Julia C API
│   └── init.h           # Helpers for Julia initialization
└── test.js            # Extension test

Requirements

• Node.js v10.16+ (N-API v4)
• Julia v1.4 (only for building the extension)
• x86-64 Linux or macOS; Windows is not (yet) supported due to build issues (PR are welcome!)

Running the extension

Installation

# Requires Node v10.16+ (N-API >= 4).
# Windows is not supported (yet).

# Pre-built binaries are avaiable for Linux and macOS (x86-64).
# If you are on a different architecture, node-pre-gyp will trigger a build.
npm install @maxmouchet/julia-node-extension-demo

Example

const clustering = require('@maxmouchet/julia-node-extension-demo')
// X: dxn matrix (n d-dimensional data points) in column-major order
// k: number of clusters
clustering.kmeans(X: Float64Array, d: Number, k: Number)

See test.js and example/ for more complete examples.

Building the extension

npm install node-pre-gyp

# Run build.jl and compile binding.cc
# => dist/linux-x64-napi-v4/binding.node
./node_modules/node-pre-gyp/bin/node-pre-gyp build

# Package everything together
# => build/stage/julia-node-extension-demo/vx.y.z/Release/linux-x64-napi-v4.tar.gz
./node_modules/node-pre-gyp/bin/node-pre-gyp package

# Upload the package to Amazon S3
./node_modules/node-pre-gyp/bin/node-pre-gyp publish

Resources

• Julia Documentation: Embedding
• julia.h and jlapi.c source files to browse the Julia C API.
• node-addon-api usage with node-gyp
• node-pre-gyp documentation
• julia #32614 - Make it easier for embedded Julia to use a custom system image
• julia #28886 - Test and document embedding with dynamically loaded libjulia.

Project Status

The package is tested against Julia 1.4 and Node.js v10/v14, on Ubuntu and macOS.