@microblink/blinkid-imagecapture-in-browser-sdk

v6.0.0

Published

a year ago

A smart image capturing library for WebAssembly-enabled browsers.

Downloads

382

0High
0Medium
0Low

BlinkID ImageCapture In-browser SDK

A frontend solution for in-browser ID scanning

With BlinkID ImageCapture In-browser SDK, your users can scan their identity documents straight from their web browser. The SDK acts as a frontend component for Microblink's BlinkID Cloud API and BlinkID Self-hosted API. It captures a high-quality image of an identity document inside a web browser and guarantees a successful scan when the image is sent to one of our backend services for processing.

The SDK offers a number of benefits that will help you build a more seamless onboarding experience for your users:

An intuitive UI that guides your users through the scanning process with error messages and real-time feedback
Built-in machine learning models that analyse the video stream from a user’s camera feed to:
- Detect the type of document in front of the lens
- Pick out the clearest frame for subsequent data extraction
A direct image upload or an option to scan a still image
A simplified communication with our Cloud or Self-hosted APIs that ensures best success rate

For more information on how to integrate BlinkID ImageCapture SDK into your web app, read the instructions below. Make sure you read the latest CHANGELOG.md file to see the most recent changes and improvements.

Check out live examples of BlinkID ImageCapture SDK in action:

BlinkID ImageCapture SDK with built-in UI
- See what the bare UI looks like at Codepen
BlinkID ImageCapture SDK UI - scan both document sides
- See example for scanning of both document sides at Codepen
Scan the front side of an identity document with a web camera
- See example at Codepen
Scan the front side of an identity document by uploading its image
- See example at Codepen
Scan both sides of an identity document with a web camera
- See example at Codepen
Scan both sides of an identity document by uploading its image
- See example at Codepen

To see the source code of the above examples, check out the examples directory. If you'd like to run examples of the UI component, either through the browser or locally, see the ui/examples directory.

Please keep in mind that BlinkID ImageCapture In-browser SDK is meant to be used natively in a web browser. It will not work correctly within a iOS/Android WebView or NodeJS backend service.

Components of SDK

BlinkID ImageCapture In-browser SDK consists of:

WASM library that recognizes a document a user is holding and extracts an image of the most suitable frame from the camera feed.
Web component with a prebuilt and customizable UI, which acts as a wrapper for the WASM library to provide a straightforward integration.
JS client library that communicates for a simple communication with backend services.

You can add it to your website or web app in two ways:

For the simplest form of integration, use a web component with a prebuilt and customizable UI.
- Follow the integration instructions in the ui/README.md file.
- You can find the source code of example applications in the ui/examples directory.
  - Note that these examples are using a client library to communicate with backend services.
For an advanced form of integration where UI has to be built from scratch, use a WASM library instead.
- See the integration instructions here.
- Find the source code of example applications in the examples directory.
  - Note that these examples are using a client library to communicate with backend services.

In both cases, we recommend using the client library to communicate with BlinkID Cloud API or BlinkID Self-hosted API backend services. Although you're free to structure the communication between the frontend and backend any way you want, since all components are decoupled.

Integration instructions

This repository contains WebAssembly files and supporting JS files which contain the core implementation of BlinkID ImageCapture functionalities.

In order to make integration of the WebAssembly easier and more developer friendly, a JavaScript/TypeScript support code is also provided, giving you an easy-to-use integration API.

This repository also contains a sample JS/TS integration app which demonstrates how you can integrate the BlinkID ImageCapture into your web app.

BlinkID ImageCapture will work in any browser that supports WebAssembly, but works best with the latest versions of Firefox, Chrome, Safari and Microsoft Edge. It's worth noting that scan performance depends on the device processing capabilities.

Obtaining a license key

Using BlinkID ImageCapture in your web app requires a valid license key.

A valid license key is required to initialize scanning. You can request a free trial license key, after you register, at Microblink Developer Hub.

Make sure you enter a fully qualified domain name of your web app when filling out the form — the license key will be bound to it. Also, if you plan to serve your web app from different domains, you'll need a license key for each one.

Keep in mind: Versions BlinkID ImageCapture 5.8.0 and above require an internet connection to work under our new License Management Program.

This means your web app has to be connected to the Internet in order for us to validate your trial license key. Scanning or data extraction of documents still happens offline, in the browser itself.

Once the validation is complete, you can continue using the SDK in an offline mode (or over a private network) until the next check.

We've added error callback to Microblink SDK to inform you about the status of your license key.

Installation

We recommend you install a stable version via NPM or Yarn:

# NPM
npm install @microblink/blinkid-imagecapture-in-browser-sdk

# Yarn
yarn add @microblink/blinkid-imagecapture-in-browser-sdk

Which can then be used with a module bundler in Node environment:

import * as BlinkIDImageCaptureSDK from "@microblink/blinkid-imagecapture-in-browser-sdk";

Source code of BlinkIDImageCaptureSDK is written in TypeScript and types are exposed in the public NPM package, so it's possible to use the SDK in both JavaScript and TypeScript projects.

Alternatively, it's possible to use UMD builds which can be loaded from public CDN services.

However, we strongly advise that you host the JavaScript bundles on your infrastructure since there is no guarantee that the public CDN service has satisfactory uptime and availability throughout the world.

For example, it's possible to use UMD builds from the dist folder on the jsDelivr CDN. The UMD builds make BlinkIDImageCaptureSDK available as a window.BlinkIDImageCaptureSDK global variable:

<!-- IMPORTANT: change "X.Y.Z" to the version number you wish to use! -->
<script src="https://cdn.jsdelivr.net/npm/@microblink/[email protected]/dist/blinkid-imagecapture-sdk.min.js"></script>

Finally, it's possible to use ES builds, which can be downloaded from the es folder on jsDelivr. ES modules are used in a similar manner as NPM package:

import * as BlinkIDImageCaptureSDK from "./es/blinkid-imagecapture-sdk.js";

Important: the jsDelivr CDN is used here due to simplicity of usage. It's not intended to be used in production!

WASM Resources

After adding BlinkID ImageCapture SDK to your project, make sure to include all files from its resources folder in your distribution. Those files contain a compiled WebAssembly module and support JS code.

Do not add those files to the main app bundle, but rather place them on a publicly available location so that the SDK can load them at an appropriate time. For example, place the resources in my-angular-app/src/assets/ folder if using ng new or in my-react-app/public/ folder if using create-react-app.

For more information on how to setup aforementioned resources, check out the Configuration of SDK section.

Versions and backward compatibility

Even though the API is not going to change between minor versions, the structure of results for various recognizers might change between minor versions.

This is due to the improvements we make to our recognizers with every minor release. We suggest you familiarize yourself with what Recognizer, RecognizerRunner and VideoRecognizer are before moving on.

It's a good practice to always lock your minor version and check the CHANGELOG.md file before upgrading to a new minor version.

For example, in package.json you should have something like "@microblink/blinkid-imagecapture-in-browser-sdk": "~4.1.1" instead of the default "@microblink/blinkid-imagecapture-in-browser-sdk": "^4.1.1".

Performing your first scan

Note: the following code snippets are written in TypeScript, but it's possible to use them in plain JavaScript.

Make sure you have a valid license key. See Obtaining a license key.
Add the SDK to your web app by using one of the options provided in the Installation section.

Initialize the SDK using the following code snippet:

import * as BlinkIDImageCaptureSDK from "@microblink/blinkid-imagecapture-in-browser-sdk";

// Check if browser is supported
if ( BlinkIDImageCaptureSDK.isBrowserSupported() )
{
    const loadSettings = new BlinkIDImageCaptureSDK.WasmSDKLoadSettings( "your-base64-license-key" );

    BlinkIDImageCaptureSDK.loadWasmModule( loadSettings ).then
    (
        ( wasmSDK: BlinkIDImageCaptureSDK.WasmSDK ) =>
        {
            // The SDK was initialized successfully, save the wasmSDK for future use
        },
        ( error: any ) =>
        {
            // Error happened during the initialization of the SDK
            console.log( "Error during the initialization of the SDK!", error );
        }
    )
}
else
{
    console.log( "This browser is not supported by the SDK!" );
}

Create recognizer objects that will perform image recognition, configure them to your needs (to scan specific types of documents, for example) and use them to create a RecognizerRunner object:

import * as BlinkIDImageCaptureSDK from "@microblink/blinkid-imagecapture-in-browser-sdk";

const recognizer = await BlinkIDImageCaptureSDK.createBlinkIdImageCaptureRecognizer( wasmSDK );
const recognizerRunner = await BlinkIDImageCaptureSDK.createRecognizerRunner(
    wasmSDK,
    [ recognizer ],
    true
);

Obtain a reference to your HTML video element and create a VideoRecognizer using the element and your instance of RecognizerRunner which then can be used to process input video stream:

const cameraFeed = document.getElementById( "myCameraVideoElement" ) as HTMLVideoElement;
try
{
    const videoRecognizer = await BlinkIDImageCaptureSDK.VideoRecognizer.createVideoRecognizerFromCameraStream(
        cameraFeed,
        recognizerRunner
    );

    // There is more than one way to handle recognition

    // Using the recognize() method will provide you with the default behavior,
    // such as built-in error handling, timeout and video feed pausing.
    const processResult = await videoRecognizer.recognize();

    // Using the startRecognition() method allows you to pass your own onScanningDone callback, 
    // giving you the option to create custom behavior.
    const processResult = await videoRecognizer.startRecognition(
        async ( recognitionState ) => 
        {
            videoRecognizer.pauseRecognition();
            return recognitionState;
        }
    );

    // To obtain recognition results see next step
}
catch ( error )
{
    if ( error.name === "VideoRecognizerError" )
    {
        // Reason is of type BlinkIDImageCaptureSDK.NotSupportedReason and contains information why video
        // recognizer could not be used. Usually this happens when user didn't grant access to a
        // camera or when a hardware or OS error occurs.
        const reason = ( error as BlinkIDImageCaptureSDK.VideoRecognizerError ).reason;
    }
}

If processResult returned from VideoRecognizer's method recognize or startRecognition is not BlinkIDImageCaptureSDK.RecognizerResultState.Empty, then at least one recognizer given to the RecognizerRunner above contains a recognition result. You can extract the result from each recognizer using its getResult method:
```
if ( processResult !== BlinkIDImageCaptureSDK.RecognizerResultState.Empty )
{
    const recognitionResult = await recognizer.getResult();
    console.log( recognitionResult );
}
else
{
    console.log( "Recognition was not successful!" );
}
```
Finally, release the memory on the WebAssembly heap by calling delete method on both RecognizerRunner and each of your recognizers. Also, release the camera stream by calling releaseVideoFeed on instance of VideoRecognizer:
```
videoRecognizer.releaseVideoFeed();
recognizerRunner.delete();
recognizer.delete();
```
Note that after releasing those objects it is not valid to call any methods on them, as they are literally destroyed. This is required to release memory resources on WebAssembly heap which are not automatically released with JavaScript's garbage collector. Also, note that results returned from getResult method are placed on JavaScript's heap and will be cleaned by its garbage collector, just like any other normal JavaScript object.

Recognizing still images

If you just want to perform recognition of still images and do not need live camera recognition, you can do that as well.

Initialize recognizers and RecognizerRunner as described in the steps 1-4 above.
Make sure you have the image set to a HTMLImageElement. If you only have the URL of the image that needs recognizing, you can attach it to the image element with following code snippet:
```
const imageElement = document.getElementById( "imageToProcess" ) as HTMLImageElement;
imageElement.src = URL.createObjectURL( imageURL );
await imageElement.decode();
```

Obtain the CapturedFrame object using function captureFrame and give it to the processImage method of the RecognizerRunner:

const imageFrame = BlinkIDImageCaptureSDK.captureFrame( imageElement );
const processResult = await recognizerRunner.processImage( imageFrame );

Proceed as in steps 6-7 above. Note that you don't have to release any resources of VideoRecognizer here as we were only recognizing a single image, but RecognizerRunner and recognizers must be deleted using the delete method.

Configuration of SDK

You can modify the default behaviour of the SDK before a WASM module is loaded.

Check out the following code snippet to learn how to configure the SDK and which non-development options are available:

// Create instance of WASM SDK load settings
const loadSettings = new BlinkIDImageCaptureSDK.WasmSDKLoadSettings( "your-base64-license-key" );

/**
 * Write a hello message to the browser console when license check is successfully performed.
 *
 * Hello message will contain the name and version of the SDK, which are required information for all support
 * tickets.
 *
 * The default value is true.
 */
loadSettings.allowHelloMessage = true;

/**
 * Absolute location of WASM and related JS/data files. Useful when resource files should be loaded over CDN, or
 * when web frameworks/libraries are used which store resources in specific locations, e.g. inside "assets" folder.
 *
 * Important: if the engine is hosted on another origin, CORS must be enabled between two hosts. That is, server
 * where engine is hosted must have 'Access-Control-Allow-Origin' header for the location of the web app.
 *
 * Important: SDK and WASM resources must be from the same version of a package.
 *
 * Default value is empty string, i.e. "". In case of empty string, value of "window.location.origin" property is
 * going to be used.
 */
loadSettings.engineLocation = "";

/**
 * The absolute location of the Web Worker script file that loads the WebAssembly module.
 *
 * Important: the worker script must be served via HTTPS and must be of the same origin as the initiator.
 * See https://github.com/w3c/ServiceWorker/issues/940 (same applies for Web Workers).
 *
 * Important: SDK, worker script and WebAssembly resources must be from the same version of the package.
 *
 * The default value is an empty string, i.e. "", and in that case, the worker script is loaded from the default location in resources folder.
 */
loadSettings.workerLocation = "";

/**
 * Type of the WASM that will be loaded. By default, if not set, the SDK will automatically determine the best WASM
 * to load.
 */
loadSettings.wasmType: WasmType | null = null;

/**
 * Optional callback function that will report the SDK loading progress.
 *
 * This can be useful for displaying progress bar to users with slow connections.
 *
 * The default value is "null".
 *
 * @example
 * loadSettings.loadProgressCallback = (percentage: number) => console.log(`${ percentage }% loaded!`);
 */
loadSettings.loadProgressCallback = null;

// After load settings are configured, proceed with the loading
BlinkIDImageCaptureSDK.loadWasmModule( loadSettings ).then( ... );

There are some additional options which can be seen in the configuration class WasmLoadSettings.

Deployment guidelines

This section contains information on how to deploy a web app which uses BlinkID ImageCapture In-browser SDK.

HTTPS

Make sure to serve the web app over a HTTPS connection.

Otherwise, the browser will block access to a web camera and remote scripts due to security policies.

Deployment of WASM files

WASM wrapper contain three different builds:

Basic
- The WASM that will be loaded will be most compatible with all browsers that support the WASM, but will lack features that could be used to improve performance.
Advanced
- The WASM that will be loaded will be built with advanced WASM features, such as bulk memory, SIMD, non-trapping floating point and sign extension. Such WASM can only be executed in browsers that support those features. Attempting to run this WASM in a non-compatible browser will crash your app.
AdvancedWithThreads
- The WASM that will be loaded will be build with advanced WASM features, just like above. Additionally, it will be also built with support for multi-threaded processing. This feature requires a browser with support for both advanced WASM features and SharedArrayBuffer.
- For multi-threaded processing there are some things that needs to be set up additionally, like COOP and COEP headers, more info about web server setup can be found here.
- Keep in mind that this WASM bundle requires that all resources are on the same origin. So, for example, it's not possible to load WASM files from some CDN. This limitation exists due to browser security rules.

Files: resources/{basic,advanced,advanced-threads}/BlinkID ImageCaptureWasmSDK.{data,js,wasm}

Server Configuration

If you know how WebAssembly works, then you'll know a browser will load the .wasm file it needs to compile it to the native code. This is unlike JavaScript code, which is interpreted and compiled to native code only if needed (JIT, a.k.a. Just-in-time compilation). Therefore, before BlinkID ImageCapture is loaded, the browser must download and compile the provided .wasm file.

In order to make this faster, you should configure your web server to serve .wasm files with Content-Type: application/wasm. This will instruct the browser that this is a WebAssembly file, which most modern browsers will utilize to perform streaming compilation, i.e. they will start compiling the WebAssembly code as soon as first bytes arrive from the server, instead of waiting for the entire file to download.

For more information about streaming compilation, check this article from MDN.

If your server supports serving compressed files, you should utilize that to minimize the download size of your web app. It's easy to notice that .wasm file is not a small file, but it is very compressible. This is also true for all other files that you need to serve for your web app.

For more information about configuring your web server to compress and optimally deliver BlinkID ImageCapture SDK in your web app, see the official Emscripten documentation.

Location of WASM and related support files

You can host WASM and related support files in a location different from the one where your web app is located.

For example, your WASM and related support files can be located in https://cdn.example.com, while the web app is hosted on https://example.com.

In that case it's important to set CORS headers in response from https://cdn.example.com. i.e. set header Access-Control-Allow-Origin with proper value so that the web page knows it’s okay to take on the request.

If WASM engine folders are not placed in the same folder as web app, don't forget to configure instance of WasmSDKLoadSettings with proper location:

...
const loadSettings = new BlinkIDImageCaptureSDK.WasmSDKLoadSettings( licenseKey );

loadSettings.engineLocation = "https://cdn.example.com/wasm";
...

The location should point to folder containing folders basic, advanced and advanced-threads that contain the WebAssembly and its support files.

The difference between basic, advanced and advanced-threads folders are in the way the WebAssembly file was built:

WebAssembly files in basic folder were built to be most compatible, but less performant.
WebAssembly files in advanced folder can yield better scanning performance, but requires more modern browser
WebAssembly files in the advanced-threads folder uses advanced WASM features as the WASM in the advanced folder but will additionally use WebWorkers for multi-threaded processing which will yield best performance.

Depending on what features the browser actually supports, the correct WASM file will be loaded automatically.

Note that in order to be able to use WASM from the advanced-threads folder, you need to configure website to be "cross-origin isolated" using COOP and COEP headers, as described in this article. This is required for browser to allow using the SharedArrayBuffer feature which is required for multi-threaded processing to work. Without doing so, the browser will load only the single-threaded WASM binary from the advanced folder.

# NGINX web server COEP and COOP header example

...

server {
    location / {
        add_header Cross-Origin-Embedder-Policy: require-corp;
        add_header Cross-Origin-Opener-Policy: same-origin;
    }
}

...

Setting up multiple licenses

As mentioned, the license key of BlinkID ImageCapture SDK is tied to your domain name, so it's required to initialize the SDK with different license keys based on the location of your web app.

A common scenario is to have different license keys for development on the local machine, staging environment and production environment. Our team will be happy to issue multiple trial licenses if needs be. See Obtaining a license key.

There are two most common approaches regarding setup of your license key(s):

Multiple apps: build different versions of your web app for different environments
Single app: build a single version of your web app which has logic to determine which license key to use

Multiple apps

Common approach when working with modern frameworks/libraries.

Single app

Simple approach, where handling of license key is done inside the web app.

Here is one possible solution:

let licenseKey = "..."; // Place your development license key here

if ( window.location.hostname === "staging.example.com" ) // Place your staging domain here
{
    licenseKey = "..."; // Place your staging license key here
}

if ( window.location.hostname === "example.com" ) // Place your production domain here
{
    licenseKey = "..."; // Place your production license key here
}
...

The `Recognizer` concept, `RecognizerRunner` and `VideoRecognizer`

This section will first describe what a Recognizer is and how it should be used to perform recognition of images, videos and camera stream. We'll also describe what RecognizerRunner is and how it can be used to tweak the recognition procedure. Finally, we'll describe what VideoRecognizer is and explain how it builds on top of RecognizerRunner in order to provide support for recognizing a video or a camera stream.

The `Recognizer` concept

The Recognizer is the basic unit tasked with reading documents within the domain of BlinkID ImageCapture SDK. Its main purpose is to process the image and extract meaningful information from it. As you will see later, BlinkID ImageCapture SDK has lots of different Recognizer objects you can set up to recognize various documents.

The Recognizer is the object on the WebAssembly heap, which means that it will not be automatically cleaned up by the garbage collector once it's not required anymore. Once you are done using it, you must call the delete method on it to release the memory on the WebAssembly heap. Failing to do so will result in memory leak on the WebAssembly heap which may result in a crash of the browser tab running your web app.

Each Recognizer has a Result object, which contains the data that was extracted from the image. The Result for each specific Recognizer can be obtained by calling its getResult method, which will return a Result object placed on the JS heap, i.e. managed by the garbage collector. Therefore, you don't need to call any delete-like methods on the Result object.

Every Recognizer is a stateful object that can be in two possible states: idle state and working state.

While in idle state, you are allowed to call method updateSettings which will update its properties according to the given settings object. At any time, you can call its currentSettings method to obtain its currently applied settings object.

After you create a RecognizerRunner with an array containing your recognizer, the state of the Recognizer will change to working state, in which Recognizer object will be used for processing. While being in working state, it is not possible to call method updateSettings (calling it will crash your web app).

If you need to change configuration of your recognizer while it's being used, you need to:

Call its currentSettings method to obtain its current configuration
Update it as you need it
Create a new Recogizer of the same type
Call updateSettings on it with your modified configuration
Replace the original Recognizer within the RecognizerRunner by calling its reconfigureRecognizers method

When written as a pseudocode, this would look like:

import * as BlinkIDImageCaptureSDK from "@microblink/blinkid-imagecapture-in-browser-sdk";

// Assume myRecognizerInUse is used by the recognizerRunner
const currentSettings = await myRecognizerInUse.currentSettings();

// Modify currentSettings as you need
const newRecognizer = await BlinkIDImageCaptureSDK.createRecognizer(); // use appropriate recognizer creation function
await newRecognizer.updateSettings( currentSettings );

// Reconfigure recognizerRunner
await recognizerRunner.reconfigureRecognizers( [ newRecognizer ], true ); // use `true` or `false` depending of what you want to achieve (see below for the description)

// newRecognizer is now in use and myRecognizerInUse is no longer in use -
// you can delete it if you don't need it anymore
await myRecognizerInUse.delete();

While Recognizer object works, it changes its internal state and its result. The Recognizer object's Result always starts in Empty state. When corresponding Recognizer object performs the recognition of a given image, its Result can either stay in Empty state (in case Recognizer failed to perform recognition), move to Uncertain state (in case Recognizer performed the recognition, but not all mandatory information was extracted) or move to Valid state (in case Recognizer performed recognition and all mandatory information was successfully extracted from the image).

`RecognizerRunner`

The RecognizerRunner is the object that manages the chain of individual Recognizer objects within the recognition process.

It must be created by createRecognizerRunner method of the WasmModuleProxy interface, which is a member of WasmSDK interface which is resolved in a promise returned by the loadWasmModule function you've seen above. The function requires two parameters: an array of Recognizer objects that will be used for processing and a boolean indicating whether multiple Recognizer objects are allowed to have their Results enter the Valid state.

To explain the boolean parameter further, we first need to understand how RecognizerRunner performs image processing.

When the processImage method is called, it processes the image with the first Recognizer in the chain. If Recognizer's Result object changes its state to Valid, and if the above boolean parameter is false, the recognition chain will be stopped and Promise returned by the method will be immediately resolved. If the above parameter is true, then the image will also be processed with other Recognizer objects in chain, regardless of the state of their Result objects.

That means if after processing the image with the first Recognizer in the chain, its Result object's state is not changed to Valid, the RecognizerRunner will use the next Recognizer object in chain for processing the image and so on - until the end of the chain (if no results become valid or always if above parameter is true) or until it finds the recognizer that has successfully processed the image and changed its Result's state to Valid (if above parameter is false).

You cannot change the order of the Recognizer objects within the chain - regardless of the order in which you give Recognizer objects to RecognizerRunner (either to its creation function createRecognizerRunner or to its reconfigureRecognizers method), they are internally ordered in a way that ensures the best performance and accuracy possible.

Also, in order for BlinkID ImageCapture SDK to be able to sort Recognizer objects in the recognition chain the best way, it is not allowed to have multiple instances of Recognizer objects of the same type within the chain. Attempting to do so will crash your application.

Performing recognition of video streams using `VideoRecognizer`

Using RecognizerRunner directly could be difficult in cases when you want to perform recognition of the video or the live camera stream. Additionally, handling camera management from the web browser can be sometimes challenging. In order to make this much easier, we provided a VideoRecognizer class.

To perform live camera recognition using the VideoRecognizer, you will need an already configured RecognizerRunner object and a reference to HTMLVideoElement to which camera stream will be attached.

To perform the recognition, you should simply write:

const cameraFeed = <HTMLVideoElement> document.getElementById( "cameraFeed" );
try
{
    const videoRecognizer = await BlinkIDImageCaptureSDK.VideoRecognizer.createVideoRecognizerFromCameraStream(
        cameraFeed,
        recognizerRunner
    );
    const processResult = await videoRecognizer.recognize();
}
catch ( error )
{
    // Handle camera error
}

The recognize method of the VideoRecognizer will start the video capture and recognition loop from the camera and will return a Promise that will be resolved when either processImage of the given RecognizerRunner returns Valid for some frame or the timeout given to recognize method is reached (if no timeout is given, a default one is used).

Recognizing a video file

If, instead of performing recognition of live video stream, you want to perform recognition of a pre-recorded video, you should simply construct VideoRecognizer using a different function, as shown below:

const videoRecognizer = await BlinkIDImageCaptureSDK.createVideoRecognizerFromVideoPath(
    videoPath,
    htmlVideoElement,
    recognizerRunner
);
const processResult = await videoRecognizer.recognize();

Custom UX with `VideoRecognizer`

The procedure for using VideoRecognizer described above is quite simple, but has some limits. For example, you can only perform one shot scan with it. As soon as the promise returned by recognize method resolves, the camera feed is paused and you need to start new recognition.

However, if you need to perform multiple recognitions in single camera session, without pausing the camera preview, you can use the startRecognition method, as described in the example below:

videoRecognizer.startRecognition
(
    ( recognitionState: BlinkIDImageCaptureSDK.RecognizerResultState ) =>
    {
        // Pause recognition before performing any async operation - this will make sure that
        // recognition will not continue while returning the control flow back from this function.
        videoRecognizer.pauseRecognition();

        // Obtain recognition results directly from recognizers associated with the RecognizerRunner
        // that is associated with the VideoRecognizer

        if ( shouldContinueScanning )
        {
            // Resume recognition
            videoRecognizer.resumeRecognition( true );
        }
        else
        {
            // Pause the camera feed
            videoRecognizer.pauseVideoFeed();
            // After this line, the VideoRecognizer is in the same state as if promise returned from
            // recognizer was resolved
        }
        // If videoRecognizer is not paused or terminated, after this line the recognition will
        // continue and recognition state will be retained
    }
);

Handling processing events with `MetadataCallbacks`

Processing events, also known as Metadata callbacks are purely intended to provide users with on-screen scanning guidance or to capture some debug information during development of your web app using BlinkID ImageCapture SDK.

Callbacks for all events are bundled into the MetadataCallbacks object. We suggest that you have a look at the available callbacks and events which you can handle in the source code of the MetadataCallbacks interface.

You can link the MetadataCallbacks interface with RecognizerRunner either during creation or by invoking its method setMetadataCallbacks. Please note that both those methods need to pass information about available callbacks to the native code. For efficiency reasons this happens at the time setMetadataCallbacks is called, not every time a change occurs within the MetadataCallbacks object.

This means that if you, for example, set onQuadDetection to MetadataCallbacks after you already called setMetadataCallbacks method, the onQuadDetection will not be registered with the native code and therefore it will not be called.

Similarly, if you remove the onQuadDetection from MetadataCallbacks object after you already called setMetadataCallbacks method, your app will crash in attempt to invoke a non-existing function when our processing code attempts to invoke it. We deliberately do not perform null check here because of two reasons:

It is inefficient
Having no callback, while still being registered to native code is illegal state of your program and it should therefore crash

Remember that whenever you make some changes to the MetadataCallbacks object, you need to apply those changes to your RecognizerRunner by calling its setMetadataCallbacks method.

List of available recognizers

This section will give a list of all Recognizer objects that are available within BlinkID ImageCapture SDK, their purpose and recommendations on how they should be used to achieve best performance and user experience.

BlinkID ImageCapture recognizer

The BlinkIdImageCaptureRecognizer is a recognizer that recognizes a document a user is holding and extracts an image of the most suitable frame from the camera feed.

Also, this recognizer can process still images for which it will determine document class and if the image is good enough for processing on the backend service.

Filter documents by class

One useful object that is returned by BlinkIdImageCaptureRecognizer is class info. This object contains information about:

Document type
Country
Region

Values from this object can be compared to enum values in ClassInfo.ts file.

Recognizer settings

It's possible to enable various recognizer settings before recognition process to modify default behaviour of the recognizer.

List of all recognizer options is available in the source code of each recognizer, while list of all recognizers is available in the List of available recognizers section.

Recognizer settings should be enabled right after the recognizer has been created in the following manner:

// Create instance of recognizer
const BlinkIdImageCaptureRecognizer = await BlinkIDImageCaptureSDK.createBlinkIdImageCaptureRecognizer( sdk );

// Retrieve current settings
const settings = await BlinkIdImageCaptureRecognizer.currentSettings();

// Update desired settings
settings[ " <recognizer_available_setting> " ] = true;

// Apply settings
await BlinkIdImageCaptureRecognizer.updateSettings( settings );

...

Technical requirements

This document provides information about technical requirements of end-user devices to run BlinkID ImageCapture.

Requirements:

The browser is supported.
The browser has access to camera device.
The device has enough computing power to extract data from an image.

Important: BlinkID ImageCapture may not work correctly in WebView/WKWebView/SFSafariViewController. See this section.

Supported browsers

Minimal browser versions with support for all features required by BlinkID ImageCapture.

|Chrome|Safari|Edge|Firefox|Opera|iOS Safari|Android Browser|Chrome for Android|Firefox for Android| |------|------|----|-------|-----|----------|---------------|------------------|-------------------| | 96| 15| 93| 79| 82| 15| 81| 96| 79|

Internet Explorer is not supported.

Sources: caniuse and WebAssembly Roadmap

Camera devices

Keep in mind that camera device is optional, since BlinkID ImageCapture can extract data from still images.

SDK cannot access camera on iOS 14.2 and older versions when the end-user is using a web browser other than Safari. Apple does not allow access to camera via WebRTC specification for other browsers.

Notes & Guidelines

For optimal data extraction use high-quality camera device in well-lit space and don't move the camera too much.
It's recommended to use camera devices with autofocus functionality for fastest data extraction.
Camera devices on MacBook laptops don't work well with low ambient light, i.e. scanning will take longer than usual.

Device support

It's hard to pinpoint exact hardware specifications for successful data extraction, but based on our testing mid-end and high-end smartphone devices released in 2018 and later should be able to extract data from an image in a relatively short time frame.

Notes & Guidelines

Browsers supported by BlinkID ImageCapture can run on older devices, where extraction can take much longer to execute, e.g. around 30 or even 40 seconds.

SDK and WebView/WKWebView/SFSafariViewController

Android and WebView

WebView is not supported for a couple of reasons:

There is no guarantee that developers of mobile apps are using WebView with all necessary features enabled.
It's up to developers of mobile apps to provide support for camera access from WebView (which is integral part of our experience), which requires additional work compared to classic camera permission in mobile apps.

Also, it's possible for mobile app developers to use WebView alternatives like GeckoView and similar, which have their own constraints.

iOS, WKWebView and SFSafariViewController

As for now, it's not possible to access the camera from WKWebView and SFSafariViewController.

Camera access on iOS, i.e. WebRTC, is only supported in Safari browser. Other browsers like Chrome and Firefox won't work as expected.

Conclusion

There is a general technical constraint when using BlinkID ImageCapture from in-app browser - it's not possible to know for sure if the SDK has or hasn't got camera access. That is, it's not possible to notify the user if the camera is not available during the initialization.

However, majority of widely used apps with in-app browsers, e.g. Facebook and Snapchat, are using standard WebView or embedded Safari with all the features. For example, WASM and modern JS are supported.

But the major problem still remains, how to get an image from the camera? Currently, we can advise two approaches:

Detect via UA string if in-app browser is used and prompt the user to use the native browser.
Detect via UA string if in-app browser is used and enable classic image upload via <input type="file" accept="image/*" capture="environment" /> element.
- Based on the operating system and software version, users will be able to select an image from the gallery, or to capture an image from the camera.

Troubleshooting

Integration problems

In case you're having issues integrating our SDK, the first thing you should do is revisit our integration instructions and make sure to closely follow each step.

If you have followed the instructions to the letter and you still have problems, please contact us at help.microblink.com.

When contacting us, please make sure you include the following information:

Log from the web console.
High resolution scan/photo of the document that you are trying to scan.
Information about the device and browser that you are using — we need the exact version of the browser and operating system it runs on. Also, if it runs on a mobile device, we also need the model of the device in question (camera management is specific to browser, OS and device).
Please stress out that you are reporting a problem related to the WebAssembly version of the BlinkID ImageCapture SDK.

SDK problems

In case of problems with using the SDK, you should do as follows:

Licensing problems

If you are getting an "invalid license key" error or having other license-related problems (e.g. some feature is not enabled that should be), first check the browser console. All license-related problems are logged to the web console so that it's easier to determine what went wrong.

When you can't determine the license-related problem or you simply do not understand the log information, you should contact us at help.microblink.com. When contacting us, please make sure you provide following information:

Exact fully qualified domain name of your app, i.e. where the app is hosted.
License that is causing problems.
Please stress out that you are reporting a problem related to the WebAssembly version of the BlinkID ImageCapture SDK.
If unsure about the problem, you should also provide an excerpt from the web console containing the license error.

FAQ and known issues

After switching from trial to production license I get error This entity is not allowed by currently active license! when I create a specific Recognizer object.

Each license key contains information about which features are allowed to use and which are not. This error indicates that your production license does not allow the use of a specific Recognizer object. You should contact support to check if the provided license is OK and that it really contains the features you've requested.

Why am I getting No internet connection error if I'm on a private network?

Versions BlinkID ImageCapture 5.8.0 and above require an internet connection to work under our new License Management Program.