mtr-sdk

Contents

• Description
• Usage
• Features
• Demo

Description

The current repository is the MTR - SDK project. The system is responsible to retrieve information from mtr.center API and load the AR in a canvas.

Usage

In order to use the module, first you'll need the mtr-sdk script loaded in html with:

<script
  id="mtr-sdk"
  type="module"
  src="https://unpkg.com/mtr-sdk/dist/mtr_sdk.js"
></script>

After that, add the div with the name you want to load the AR, for example, id="mtr-ar-holder" and set the width and height of the canvas to show.

<div id="mtr-ar-holder" style="width: 390px; height: 720px;"></div>

And then, you can load the SDK with the following script.

const config = {
  projectId: 146,
  divName: "mtr-ar-holder",
  verbose: true,
  loadedCallback: () => {
    console.log("SDK loaded");
  },
};
window.mtrObject = new MTR(config);

Where config is an object with properties:

• projectId type number: MTR Project ID. More information about the project available on mtr.center. No default.

• divName type string: ID of the div holder to load the AR project. The ID must be the same as the one on the HTML file. Default is mtr-ar-holder.

• verbose type boolean: If set to true, shows logs on the console. Default is false.

• loadedCallback type Function. Function to call after MTR is loaded. The function will also be returned with an array of objects called infoReturn. This array is used retrieve parameter information to switch models on model viewer and meshes on face mesh. No default.

  • Model Viewer:

  infoReturn = [{objectName, variantNames, variantHtmlIds}, ...]

  • Face Mesh:

  infoReturn = [{htmlId, meshConfigsId, materialId, name, ean}, ...]

With those lines of code it's already possible to run the sdk.

Features

• .switchModelViewerModel(slotName): you can use this function in case you want to change the 3D model in Model Viewer. The name of the slot will be available at mtr.center or infoReturn. Example:

  window.mtrObject.switchModelViewerModel(infoReturn[0].objectName);

• .mtrObject.modelViewerHolder.variantName: you can also change the variant of the model by changing the variantName.

  window.mtrObject.modelViewerHolder.variantName = infoReturn[0].variantNames[0];

• .switchFaceMesh(meshConfigsId, materialId): you can use this function when you want to change the mesh on the face without using the button click. The parameters are available on infoReturn. Example:

  window.mtrObject.switchFaceMesh(infoReturn[0].meshConfigsId, infoReturn[0].materialId);

• .stop(): this function stops the application and take off the control buttons from the holder specified. Use this function if you want to stop processing face mesh.

  window.mtrObject.stop();

• .continue(): this function continues the application and put the control buttons back on the holder specified. Use this function if you want to start the face mesh process again.

  window.mtrObject.continue();

• .shareScreenshot(fileName: string): this function takes a screenshot of the AR scene and shares it.

  window.mtrObject.shareScreenshot("MyFileName");

• .modelViewerHolder.activateAR(): after calling this on devices that support AR, the model is placed on the user environment. Works like a "View in your room" button.

  window.mtrObject.modelViewerHolder.activateAR();

• retornar no loadedCallback() um json do nome do material e do id sendo "mtr-item-" os botoes
• .startFaceMesh(width: number, height: number): call requestPermissions built in function to start the camera. width and height are the dimensions of the camera.
• .colorExtraction(colorType: string = "RGB"): returns a Promise of a string or array of 3 numbers depending on colorType parameter. If colorType is not set, the default is "RGB" and the return value will be an array of 3 positions representing the color RGB of the skin color extracted, otherwise the color returned will be a hexadecimal value in the format "#ABCDEF".

Face Mesh exclusive Features

The AR object also include other functionalities.

Partial Render

The partial render works like a curtain for the threejs renderer and only renders what is from 0 to windowSlider.value * canvas.width. For this, you need to create a checkbox and a slider from 0 to 1 and then use them as follows.

const windowCheckbox = document.getElementById('window-checkbox');
const windowSlider = document.getElementById('window-slider');

windowCheckbox.oninput = function () {
  window.mtrObject.faceMesh.setWindow(windowCheckbox.checked);
  windowSlider.style.display = windowCheckbox.checked ? 'block' : 'none';
};

windowSlider.oninput = function () {
  window.mtrObject.faceMesh.setWindowFactor(windowSlider.value);
};

Photograph

The photograph functionality is simple and works like it is said. When the method togglePhoto is called, the animation loop freezes/unfreezes depending on the actual state. You only need to create on html and use the following code. The method also returns a promise of a canvas of the photograph.

const screenshotButton = document.getElementById('photo-button');

screenshotButton.onclick = async function () {
  const frozenVideoCanvas = await window.mtrObject.faceMesh.togglePhoto();
};

Foundation Color

If foundationMatchConfig and faceRetouchConfig are set, it is possible to set a color on the users face as if there is a foundation using .setFoundation method. The function receives two parameters, color and opacity. The color is a number array with 3 positions in RGB space, and the opacity is a floating point number between 0.0 and 1.0 that is the strength of the color. The following example will make up the face with red foundation.

window.mtrObject.faceMesh.setFoundation([255, 0, 0], 0.5);

Using an image instead of camera

If faceMesh is created with an image with id="mtr-photo-img" set, the video camera will be replaced with the image loaded on html. For example, an image tag is loaded with id "mtr-photo-img", and faceMesh is created with it replacing the camera.

<img id="mtr-photo-img" src="./any-image">

Demo

The demos folder shows examples of usage. The 'facemesh-photo' folder shows an application that loads a canvas on facemesh-holder div and a MTR - Face Mesh scene. There is almost no difference from this Face Mesh demo to the Model Viewer one, since the main difference will be the projectID parameter.