mtr-hair-segmentation

Contents

• Description
• Usage
• Features
• Demo

Description

The current repository is the mtr-hair-segmentation project. The module paint the users hair with the color specified.

Usage

In order to use the module, add the div with id="hair-holder" and set the width and height of the canvas to show.

<div id="hair-holder" style="width: 282px; height: 500px;"></div>

After that, you can create the HairSegmentation object with the following.

import { HairSegmentation } from "../../dist/mtr_hair_segmentation.js";

...

const config = {
  // div id
  divName: "hair-holder", // the same as the default value
  // hair color
  hairColor: [190, 3, 252, 255], // purple
  // path to the tensorflow and AI binaries
  binaryDir: "../../dist/hair_segmenter.tflite",
  // flip the camera feedback
  flipCamera: true,
  // console log every step
  verbose: true,
  // show the FPS of the animation loop
  showFPSPanel: true,
  // show the FPS of every step on the animation loop
  showScreenLogs: true,
};


hairSegmentation = new HairSegmentation(config);

Where config is an object with properties:

• divName HTML div ID of the AR holder. Default is hair-holder.
• imageID HTML image element ID that will replace the camera
• hairColor Color with the format [r, g, b, a], where rgba are values between 0-255. Default is white.
• binaryDir Path to the hair segmentation binary files. No default.
• flipCamera Flips the feedback camera. Default is false.
• verbose If set to true shows logs on the console. Default is false.
• loadedCallback Function to call after the AI model is loaded. No default.
• cameraStatusCallback Function to call after the video permission (no default):
  • requesting: when user is requested for the camera
  • hasStream: when user is granted access for the camera
  • failed: when something went wrong with the camera
• errorCallback Function to call after any error. No default.
• showFPSPanel If set to true shows the FPS on the main loop. Default is false.
• showScreenLogs If set to true shows the FPS for every step on the main loop. Default is false.

The method requestPermissions must be called after the model is loaded using the loadedCallback config property, since the hair segmentation only starts predicting after the video is set. The method receives the camera configuration as a parameter, the same as seen at getUserMedia.

const loadedCallback = function () {
  hairSegmentation.requestPermissions({
    facingMode: {ideal: 'user'},
    width: window.innerHeight,
    height: window.innerWidth,
  });
};

With these lines of code it's already possible to run the hair segmentation.

Features

The AR object also include other functionalities.

Photograph

The photograph functionality is simple and works like it is said. When the method togglePhoto is called, the animation loop freezes/unfreezes depending on the actual state. You only need to create on html and use the following code. The method also returns a promise of a canvas of the photograph.

const screenshotButton = document.getElementById('photo-button');

screenshotButton.onclick = async function () {
  const frozenVideoCanvas = await hairSegmentation.togglePhoto();
};

Share

The next feature complements the photograph feature. When the share method is called, the object takes a screenshot and then share it through the selected social media. For that, you only need to create a button to trigger the method.

const shareButton = document.getElementById('share-button');

shareButton.onclick = function () {
  hairSegmentation.share('screenshot');
};

Stop and Continue

The application can be stopped or continued after calling .stop() and .continue() methods. They are useful when you want to stop the application with a close button or start the application instantly after the user clicks a start button. The demo shows an example of usage.

const closeButton = document.getElementById('close-button');


var toggleCloseContinue = true;

closeButton.onclick = function () {
  if (toggleCloseContinue) {
    hairSegmentation.stop();
    closeButton.textContent = "Continue";
  } else {
    hairSegmentation.continue();
    closeButton.textContent = "Close";
  }
  
  toggleCloseContinue = !toggleCloseContinue;
};

Using an image instead of camera

If hairSegmentation is created with imageID set, the video camera will be replaced with the image loaded on html with id=<imageID> and .requestPermissions() method will not be necessary to call. For example, an image tag is loaded with id "photo-id", and hairSegmentation is created with it replacing the camera.

<img id="photo-id" src="./any-image">

hairSegmentation = new HairSegmentation({
  imageID: "photo-id",
  ...
});

Demo

The demos folder have demos with all the features. The photo shows an example of using an image loaded on html and the webcam uses the user camera. The application loads a canvas and shows the user with a painted hair. It is also possible to freeze the video like a photograph and share the photo.