npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@executorch/runtime-mps

v0.0.7

Published

JavaScript bindings for [ExecuTorch](https://pytorch.org/executorch-overview), a runtime for inferencing PyTorch models.

Downloads

45

Readme

ExecuTorch.js

JavaScript bindings for ExecuTorch, a runtime for inferencing PyTorch models.

Supported platforms

JavaScript runtimes:

  • Node.js >= 22 (and compatible runtimes like Electron and Bun)
  • Plan on: WebAssembly, React Native

Platforms:

  • Linux x64
  • macOS arm64
  • macOS x64
    • Some ops required for running LLMs are not supported yet.
  • Plan on: Windows, Linux arm64, iOS, Android

Backends:

Install

You can install ExecuTorch.js from npm:

$ npm install executorch

The default executorch package includes support for all backends, for users who want to reduce binary size, you can install packages with specific backeds:

| | CPU | MPS | Vulkan | XNNPACK | |-----------------------------|-----|-----|--------|---------| | executorch | ✔️ | ✔️ | ❌ | ✔️ | | @executorch/runtime | ✔️ | 🍏 | ❌ | 🐧 | | @executorch/runtime-all | ✔️ | ✔️ | ❌ | ✔️ | | @executorch/runtime-cpu | ✔️ | ❌ | ❌ | ❌ | | @executorch/runtime-mps | ✔️ | ✔️ | ❌ | ❌ | | @executorch/runtime-xnnpack | ✔️ | ❌ | ❌ | ✔️ |

The @executorch/runtime package is a speical one that uses MPS backend on macOS and XNNPACK backend for other platforms.

For debugging purpose each package also has a Debug version that can be enabled by setting the npm_config_debug environment variable when installing:

$ env npm_config_debug=true npm install @executorch/runtime-xnnpack

Quick start

Download the mobilenet model:

$ wget https://huggingface.co/frost-beta/mobilenet-v2-executorch-cpu/resolve/main/mv2.pte

Run following code with Node.js:

import {Module, Tensor} from 'executorch';

// A tensor of shape [ 1, 3, 224, 224 ].
const input = Array.from({length: 1}, () =>
              Array.from({length: 3}, () =>
              Array.from({length: 224}, () =>
              Array.from({length: 224}, () => Math.random()))));

const mod = new Module('mv2.pte');
await mod.load();
const output = await mod.forward(new Tensor(input));
console.log(output.tolist());

Examples

APIs

/**
 * Load exported edge PyTorch models.
 */
export declare class Module {
    /**
     * The methods of this class are dynamically loaded.
     */
    [key: string]: Function;
    /**
     * @param filePathOrBuffer - When a string is passed, it is treated as file
     * path and will be loaded with mmap. When a Uint8Array is passed, its content
     * is used as the model file.
     */
    constructor(filePathOrBuffer: string | Uint8Array);
    /**
     * Load the model.
     *
     * @remarks
     *
     * After loading, the model's methods will be added to the instance
     * dynamically, with both async and async versions for each method, the sync
     * version will have a "Sync" suffix appended to its name.
     */
    load(): Promise<void>;
    /**
     * Load the model synchronously.
     */
    loadSync(): void;
    /**
     * Return if any model has been loaded.
     */
    isLoaded(): boolean;
    /**
     * Return names of loaded model's methods.
     */
    getMethodNames(): string[];
}

/**
 * Data type.
 */
export declare enum DType {
    Uint8,
    Int8,
    Int16,
    Int32,
    Int64,
    Float16,
    Float32,
    Float64,
    Bool,
    BFloat16
}

type Nested<T> = Nested<T>[] | T;

/**
 * A multi-dimensional matrix containing elements of a single data type.
 */
export declare class Tensor {
    /**
     * The tensor's data stored as JavaScript Uint8Array.
     */
    readonly data: Uint8Array;
    /**
     * Data-type of the tensor’s elements.
     */
    readonly dtype: DType;
    /**
     * Array of tensor dimensions.
     */
    readonly shape: number[];
    /**
     * @param input - A scalar, or a (nested) Array, or a Uint8Array buffer.
     * @param dtype - The data type of the elements.
     * @param options - Extra information of the tensor.
     * @param options.shape
     * @param options.dimOrder
     * @param options.strides
     */
    constructor(input: Nested<boolean | number> | Uint8Array,
                dtype?: DType,
                { shape, dimOrder, strides }?: { shape?: number[]; dimOrder?: number[]; strides?: number[]; });
    /**
     * Return the tensor as a scalar.
     */
    item(): number | boolean;
    /**
     * Return the tensor as a scalar or (nested) Array.
     */
    tolist(): Nested<number | boolean>;
    /**
     * Return a TypedArray view of tensor's data.
     */
    toTypedArray(): Int8Array | Uint8Array | Int16Array | Int32Array | Float32Array | Float64Array;
    /**
     * A permutation of the dimensions, from the outermost to the innermost one.
     */
    get dimOrder(): number[];
    /**
     * Array of indices to step in each dimension when traversing the tensor.
     */
    get strides(): number[];
    /**
     * Number of tensor dimensions.
     */
    get ndim(): number;
    /**
     * Number of elements in the tensor.
     */
    get size(): number;
    /**
     * Total bytes consumed by the elements of the tensor.
     */
    get nbytes(): number;
    /**
     * Length of one tensor element in bytes.
     */
    get itemsize(): number;
}

/**
 * Samples from the given tensor using a softmax over logits.
 */
export declare function sample(logits: Tensor,
                               {
                                 temperature = 1,
                                 topP = 1,
                               }?: { temperature?: number; topP?: number }): number;

Development

Source code architecture:

  • src/ - C++ source code.
  • lib/ - TypeScript source code.
  • bindings.js/bindings.d.ts - Glue code between C++ and TypeScript.
  • install.js - Script that downloads compiled binaries when installing.
  • tests/ - Tests for TypeScript code.
  • build/ - Generated project files and binaries from C++ code.
  • dist/ - Generated JavaScript code from TypeScript code.