Handy Work

A module for doing efficient real-time pose detection from WebXR Hand Tracking using Web Workers to ensure it doesn't interfere with the main thread

The pose tracking module is Framework Agnostic, it doesn't rely on any particular library it should work just as well with THREE as BabylonJS or Play Canvas.

Poses can be found in the /poses/ folder and additional poses are welcome

There is also an AFrame module which handles pose tracking and displaying hand models.

Hand Pose Module Usage Example

import {
  update as handyWorkUpdate,
  loadPose
} from "../build/esm/handy-work.js";

loadPose('relax', '../poses/relax.handpose');
loadPose('fist', '../poses/fist.handpose');
loadPose('flat', '../poses/flat.handpose');
loadPose('point', '../poses/point.handpose');

// In RAF
handyWorkUpdate([controller1, controller2], referenceSpace, frame, callback);

Using via a CDN

If you use it via a CDN because it uses a Worker you need to use the standalone version which has the Worker encoded as a string.

import('https://cdn.jsdelivr.net/npm/[email protected]/build/esm/handy-work.standalone.js')
.then(function ({
  update,
  loadPose,
  dumpHands
}) {
  const handyWorkUpdate = update;
  const dumpHands = dumpHands;
  const loadPose = loadPose;

  loadPose('relax', POSE_FOLDER + 'relax.handpose');
  loadPose('fist', POSE_FOLDER + 'fist.handpose');
  loadPose('flat', POSE_FOLDER + 'flat.handpose');
  loadPose('point', POSE_FOLDER + 'point.handpose');
  loadPose('horns', POSE_FOLDER + 'horns.handpose');
  loadPose('shaka', POSE_FOLDER + 'shaka.handpose');
  loadPose('vulcan', POSE_FOLDER + 'vulcan.handpose');
}.bind(this));

handy-work

Documentation for using build/esm/handy-work.js or build/esm/handy-work.standalone.js

• handy-work
  • .loadPose(name, url) ⇒ Promise.<void>
  • .getPose(name) ⇒ Promise.<Float32Array>
  • .setPose(name, pose) ⇒ Promise.<void>
  • .resetHands()
  • .dumpHands()
  • .generatePose(inputSources, referenceSpace, frame) ⇒ void | Float32Array
  • .update(inputSources, referenceSpace, frame, callback)

handy-work.loadPose(name, url) ⇒ Promise.<void>

Loads a pose from the Web the recommended way of using this API

Kind: static method of handy-work
Returns: Promise.<void> - Resolves once it has been completed

| Param | Type | Description | | --- | --- | --- | | name | string | Name of the pose | | url | string | URL of the pose to download and add to the registry |

handy-work.getPose(name) ⇒ Promise.<Float32Array>

Get a pose from the registry

Kind: static method of handy-work
Returns: Promise.<Float32Array> - A copy of the pose from the registry in the worker

| Param | Type | Description | | --- | --- | --- | | name | string | of the pose to fetch |

handy-work.setPose(name, pose) ⇒ Promise.<void>

Get a pose from the registry

Kind: static method of handy-work
Returns: Promise.<void> - Resolves once the pose has been uploaded to the worker

| Param | Type | Description | | --- | --- | --- | | name | string | of the pose to set | | pose | Float32Array | Array buffer with the information about the current pose |

handy-work.resetHands()

Reset the current tracking performed automatically when the device returns from sleep

Kind: static method of handy-work

handy-work.dumpHands()

On the next frame save the current hand pose to a file

Kind: static method of handy-work

handy-work.generatePose(inputSources, referenceSpace, frame) ⇒ void | Float32Array

Get a pose from the current hand position

Kind: static method of handy-work
Returns: void | Float32Array - The generated pose buffer for the pose.

| Param | Type | Description | | --- | --- | --- | | inputSources | XRInputSource | Array of inputs you want to generate inputs for, requires a left AND right hand | | referenceSpace | XRReferenceSpace | Current reference space | | frame | XRFrame | Current active frame |

handy-work.update(inputSources, referenceSpace, frame, callback)

Kind: static method of handy-work

| Param | Type | Description | | --- | --- | --- | | inputSources | Array.<XRInputSource> | The inputs you want to do pose tracking for | | referenceSpace | XRReferenceSpace | The reference space for your scene | | frame | XRFrame | The current active frame | | callback | function | This gets called with an Array of Arrays with the poses and their distances |