deepar

v5.6.12

Published

11 days ago

The official DeepAR Web SDK

Downloads

925

0High
0Medium
0Low

deepar

deepar web sdk ar effects bacground removal bacground replacement background blur AR mini-games

deepar

DeepAR Web is an augmented reality SDK that allows users to integrate advanced, Snapchat-like face lenses in the browser environment.

DeepAR Web supports:

Glasses try-on.
Face filters and masks.
Background replacement.
Background blur.
AR mini-games.

Documentation

Visit official DeepAR docs for Web SDK here https://docs.deepar.ai/deepar-sdk/platforms/web/overview.
See the official example here: https://github.com/DeepARSDK/quickstart-web-js-npm
Set up and web ad with DeepAR: https://github.com/DeepARSDK/quickstart-web-ad
Full API reference here.

License key

In order to use the DeepAR Web SDK you need to set up a license key for your web app on developer.deepar.ai.

Create an account: https://developer.deepar.ai/signup.
Create a project: https://developer.deepar.ai/projects.
Add a web app to the project. Note that you need to specify the domain name which you plan to use for hosting the app.

⚠️ The license key property is required both in a production and development (localhost) environment. Development sessions will not count towards your monthly active usage.

Installation

Using npm:

$ npm install deepar

Using yarn:

$ yarn add deepar

Getting started

There are two main ways to get deepar.js in your JavaScript project: via script tags or by installing it from NPM and using a build tool like Parcel, WebPack, or Rollup.

via Script tag

Add the following code to an HTML file:

<html>
<head>
  <!-- Ensure compatibility and performance on mobile -->
  <meta name="viewport" content="width=device-width, initial-scale=1 "/>
  <!-- Load deepar.js -->
  <script src='https://cdn.jsdelivr.net/npm/deepar/js/deepar.js'> </script>
</head>

<body>
  <!-- Div element where AR preview will be inserted -->
  <div style='width: 640px; height: 360px' id='deepar-div'></div>
  <!-- Initialize DeepAR and load AR effect/filter -->
  <script>
    (async function() {
      const deepAR = await deepar.initialize({
        licenseKey: 'your_license_key_here',
        previewElement: document.querySelector('#deepar-div'),
        effect: 'https://cdn.jsdelivr.net/npm/deepar/effects/aviators'
      });
    })();
  </script>
</body>
</html>

Alternatively, you can import DeepAR as an ES6 module.

Via <script type='module'>.

<script type='module'>
  import * as deepar from 'https://cdn.jsdelivr.net/npm/deepar/js/deepar.esm.js';
</script>

Or via dynamic import.

const deepar = await import('https://cdn.jsdelivr.net/npm/deepar/js/deepar.esm.js');

via NPM

Add deepar.js to your project using yarn or npm.

Note: Because we use ES2017 syntax (such as import), this workflow assumes you are using a modern browser or a bundler/transpiler to convert your code to something older browsers understand. However, you are free to use any build tool that you prefer.

import * as deepar from 'deepar';

const deepAR = await deepar.initialize({
  licenseKey: 'your_license_key_here', 
  previewElement: document.querySelector('#deepar-canvas'),
  effect: 'https://cdn.jsdelivr.net/npm/deepar/effects/aviators' 
});

Switch effects

AR filters are represented by effect files in DeepAR. You can load them to preview the effect.

Places you can get DeepAR effects:

Download a free filter pack: https://docs.deepar.ai/deepar-sdk/filters#free-effects-pack-content.
Visit DeepAR asset store: https://www.store.deepar.ai/
Create your own filters with DeepAR Studio.

Load an effect using the switchEffect method:

await deepAR.switchEffect('path/to/effect/alien');

Take screenshot or video

Take a screenshot.

// The image is a data url.
const image = await deepAR.takeScreenshot();

Record a video.

await deepAR.startVideoRecording();
// Video is now recording...
// When user is ready to stop recording.
const video = await deepAR.finishVideoRecording();

Background blur

Enable background blur with strength 5.

await deepAR.backgroundBlur(true, 5)

Background replacement

This is also known as background removal or green screen effect.

Enable background replacement with an image of a sunny beach.

await deepAR.backgroundReplacement(true, 'images/sunny_beach.png')

Callbacks

DeepAR has some callbacks you can implement for additional informations. For example, to check if face is detected in the scene.

await deepAR.switchEffect('https://cdn.jsdelivr.net/npm/deepar/effects/aviators');
deepAR.callbacks.onFaceTracked = function(faceDataArr) {
    if (faceDataArr[0].detected) {
        console.log("Face is detected!");
    }
};

To remove callback if you don't need it anymore.

deepAR.callbacks.onFaceTracked = undefined;

Different video sources

DeepAR will by default use the user facing camera. It will also ask the camera permission from the user.

Use the back camera on the phones:

const deepAR = await deepar.initialize({
  // ...
  additionalOptions: {
      cameraConfig: {
          facingMode: 'environment'
      }
  }
});

Set up your own camera or custom video source:

const deepAR = await deepar.initialize({
  // ...
  additionalOptions: {
      cameraConfig: {
          disableDefaultCamera: true
      }
  }
});

// HTMLVideoElement that can contain camera or any video.
const video = ...;

deepAR.setVideoElement(video, true);

Initialize DeepAR but start the camera later. This is used when you do not want to ask the camera permission right away.

const deepAR = await deepar.initialize({
    // ...
    additionalOptions: {
        cameraConfig: {
            disableDefaultCamera: true
        }
    }
});

// When you want to ask the camera permission and start the camera.
await deepAR.startCamera();

In cases where the camera aspect ratio doesn't match the screen aspect ratio, it is possible to rotate the camera input to achieve better camera quality. This is usually the case for in-store AR screens which have portrait screens, but use standard landscape webcams. To improve camera quality in those cases, rotate the camera physically 90° and call the following API.

const deepAR = await deepar.initialize({
    // ...
    additionalOptions: {
        cameraConfig: {
            rotation: 90 // or 270
        }
    }
});

If you are using non-default DeepAR camera apply the rotation with:

deepAR.setVideoRotation(90) // or 270

Long distance face tracking (better accuracy)

In some cases, it is needed to have face tracking at longer distances, not just at hand-held distances. For that use-case, DeepAR comes with upgraded face tracking model which works better at distances and is more accurate in general. It can also be used if more accurate beauty effects are needed.

⚠️ This face tracking model works only for single person face tracking, ie. it cannot track multiple faces.

To enable this face tracking mode, pass enableFaceTrackingCnn hint at initialization.

const deepAR = await deepar.initialize({
    // ...
    additionalOptions: {
        hint: "enableFaceTrackingCnn"
    }
});

Providing your own canvas for rendering

Create canvas directly in the HTML:

<canvas width='1280' height='720'></canvas>

Or you can create it in Javascript.

const canvas = document.createElement('canvas');
// Set canvas size, accounting screen resolution (to make it look 🤌 even on high definition displays)
const canvasSize = { width: 640, height: 360 };
const dpr = window.devicePixelRatio || 1;
canvas.style.maxWidth = `${canvasSize.width}px`;
canvas.style.maxHeight = `${canvasSize.height}px`;
canvas.width = Math.floor(canvasSize.width * dpr);
canvas.height = Math.floor(canvasSize.height * dpr);

⚠️ Note: Be sure to set width and height properties of the canvas!

You can always change the canvas dimensions and they don't have to match the input video resolution. DeepAR will fit the input camera/video stream correctly to any canvas size.

You pass the canvas when initializing DeepAR.

await deepar.initialize({
  canvas: canvas, 
  // ...
});

Download speed optimizations for DeepAR Web

Apart from the main deepar.js file and AR effect files, DeepAR uses additional files like:

WebAssembly (WASM) files.
ML model files.

Some of them are required and will be downloaded every time. The others will be lazy downloaded when/if needed.

⚠️ DeepAR will by default automatically fetch all additional resources from the JsDelivr CDN.

Fetching files from JsDelivr is not recommended if you care about download speeds of DeepAR Web resources. This is because the files on JsDelivr are not compressed.

Compression

To optimize download speeds, all the DeepAR files should be compressed. It is recommended to serve DeepAR files from your own server or some CDN which supports file compression.

If it is supported, you should use GZip or Brotli compression on all DeepAR files which will significantly reduce the SDK size. Check out your server/CDN options for compressing files.

Custom deployment of DeepAR Web

DeepAR Web can be downloaded from DeepAR Developer Portal. But since most users will install DeepAR through NPM, follow the next instructions.

It is recommended to copy ./node_modules/deepar directory which contains all the DeepAR files into your distribution (dist) folder. You can use rootPath to set a path to the custom root of the DeepAR SDK. All additional files that need to be fetched by DeepAR will be resolved against the given rootPath.

const deepAR = await deepar.initialize({
  // ...
  rootPath: 'path/to/root/deepar/directory'
});

It is recommended to setup the copying of the DeepAR directory as part of you bundler build scripts, in case you ever need to updated to a newer version of DeepAR.

Specifying paths for each file

Another option, instead of using the DeepAR directory and copying it as-is, is to specify a path to each file. The advantage of this is that you can use the bundler to keep track of your files.

For example, if using Webpack, you can use it's asset modules to import all the files needed.

Pass the file paths in additionalOptions.

const deepAR = await deepar.initialize({
    // ...
    additinalOptions: {
        faceTrackingConfig: {
            modelPath: 'path/to/deepar/models/face/models-68-extreme.bin'
        },
        segmentationConfig: {
            modelPath: 'path/to/deepar/models/segmentation/segmentation-160x160-opt.bin'
        },
        deeparWasmPath: 'path/to/deepar/wasm/deepar.wasm'
    }
});

License

Please see: https://developer.deepar.ai/customer-agreement

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

deepar

Documentation

License key

Installation

Getting started

via Script tag

via NPM

Switch effects

Take screenshot or video

Background blur

Background replacement

Callbacks

Different video sources

Long distance face tracking (better accuracy)

Providing your own canvas for rendering

Download speed optimizations for DeepAR Web

Compression

Custom deployment of DeepAR Web

Specifying paths for each file

License