@onirix/ar-engine-sdk

v1.8.7

Published

5 days ago

Onirix AR Engine SDK allows to create Augmented Reality experiences that run in a web browser.

Downloads

1,175

0High
0Medium
0Low

frangaren

vifergo

onirix_development

Onirix AR Engine SDK

Onirix AR Engine SDK allows to create Augmented Reality experiencies that run in a web browser.

It internally uses advanced web standards like WebGL or WebAssembly combined with in-house computer vision algorithms optimized to run in a web browser.

Onirix AR Engine SDK is compatible with the following Onirix tracking modes:

Image: Will load the image classifier generated for your Onirix project and perform detection and tracking of any of its images.
Curved: Will load the image classifier generated for your Onirix project and perform detection and tracking of any of its images, taking into account that those images will be wrapping a cylinder.
QR Code: Will detect any QR code in the camera feed, return its decoded content and perform tracking.
Surface: Will use device camera and motion sensors to track an object fixed over any place or surface. This mode has 2 different sub-modes:
- World-Tracking (Default): Allows to place objects fixed at a surface, so you can walk around and see them from different perspectives. Internally uses WebXR device API if the device is compatible (Android with ARCore support for now), and fallbacks to a custom implementation if not, or when using the config option disableWebXR = true.
- Rotation-Tracking: Uses device gyroscope sensor to provide an accurate 360º rotation-only tracking. Useful for scenes where users are not supposed to move and rather look around. Can be enabled using the config option disableWorldTracking = true.
Spatial: Will recognize a previously scanned environment (space), positioning you within it and initiating World-Tracking from that location.

Onirix AR Engine SDK is agnostic (not tied) to any rendering engine, so you can combine it with any 3D engine library like:

ThreeJS
BabylonJS
Aframe

If you want to have an overview of how the SDK works and make a quick test you may want to go straight to the samples section.

Getting Started

Onirix Studio configuration

First of all, you'll need to access Onirix Studio and create a Project.

If you plan to use Image-Tracking or Curved-Tracking mode, then you'll also need to create an image scene for every marker image you want to be detected so Onirix can generate the required image classifier. For Spatial-Tracking, you'll also need to create an scene by scanning your environment with Onirix Constructor App. If none of these modes are used, then there is no need to create any scene as, when using the SDK, you can provide your own assets and interaction through your hosting and code.

Finally, you will have to publish your project and copy the Web SDK token from the "share" and "settings" top-menu options respectively.

Integrating with your Web App

Whether you are integrating Onirix AR Engine SDK to an existing Web App, or creating a new one from the scratch, the following steps are required:

Include Onirix AR Engine SDK

Add a custom script (app.js) to your page html head tag and load it as an async module:

<head>
 ...
    <script async type="module" src="app.js"></script>
 ...    
</head>

Inside your custom script (app.js), import Onirix SDK:

import OnirixSDK from "https://unpkg.com/@onirix/[email protected]/dist/ox-sdk.esm.js";

If you're using a JavaScript framework or a bundler like Webpack, it is recommended to use the NPM package instead.

Instantiate OnirixSDK and create a configuration object

Whenever you want to launch the AR experience, create a new OnirixSDK instance with your Onirix project's token and a configuration object with the tracking mode and other desired paramenters:

    let OX = new OnirixSDK("<your_sdk_token>");

    let config = {
        mode: OnirixSDK.TrackingMode.Image
    }

The mode can be one of following depending on the tracking type you want to use for your experience:

OnirixSDK.TrackingMode.Image
OnirixSDK.TrackingMode.Curved
OnirixSDK.TrackingMode.QRCode
OnirixSDK.TrackingMode.Surface
OnirixSDK.TrackingMode.Spatial

This is the complete list of config parameters:

| Parameter | Type | Description | | ------ | ----------- | ---------- | | mode | OnirixSDK.TrackingType (required) | The AR mode you want your experience to run with. | | disableWorldTracking | boolean (optional, default: false) | You can use this option to enable Rotation-Tracking. (Surface tracking only) | | disableWebXR | boolean (optional, default: false) | You can use this option to force using Onirix custom World Tracking implementation for more homogeneous experiences between devices and OS. (Surface and Spatial tracking only) | | useVocabulary | boolean (optional, default: false) | Enabling this setting will speed up image detection in exchange of downloading an additional file (25MB) when the experience loads. It is recommended when number of marker images per project is bigger than 10. (Image tracking only) | | sceneOid | string (required for Spatial mode) | OID of the scene in which you want to relocate |

Initialize Onirix AR Engine SDK

Call init method with prior configuration object as a parameter:

   OX.init(config).then(rendererCanvas => {
       // Onirix SDK has been initialized. Now it's time to set up a 3D renderer 
       // with any library (Three, Babylon ...) and subscribe to Onirix events.
   }).catch(error => {
       // Check error name and display accordingly
   });

When calling init, Onirix AR Engine SDK will:

Check your license
Download required resources (i.e image classifier)
Request camera and sensors access
Add a canvas for 3D rendering and a video element for camera background to your page, both adjusted to full screen size.
Return the canvas so you can initialize your desired 3D renderer library with it.

This may take some time to complete, so it is recommended to show a loading screen before calling it, and hide it when the function completes.

Note that init method returns a promise. You may want to use the new ES6 async / await syntax.

Handle errors

There are 4 types of errors that might occur when initializing Onirix AR Engine SDK. You should check the error type by inspecting its name or using instanceof JavaScript operator to compare with its corresponding error class and display in the form you decide to the user.

In the following table, you will find useful information about error types:

| Class name | Name property | Description | | ------ | ----------- | ---------- | | OnirixSDK.LicenseError | LICENSE_ERROR | Might occur if the project is not published, your account has run out of views, has expired, or website domain is not valid | | OnirixSDK.CameraError | CAMERA_ERROR | Might occur if could not detect or access to back-facing camera, or if camera permissions have been rejected | | OnirixSDK.SensorsError | SENSORS_ERROR| Might occur if could not detect or access to gyroscope sensor, or motion sensor permissions have been rejected | | OnirixSDK.InternalError | INTERNAL_ERROR| Might occur in non-compatible devices. Check compatibility for more info | | OnirixSDK.ReconstructionPendingError | RECONSTRUCTION_PENDING_ERROR| Might occur when the spatial scene has not yet finished its reconstruction |

Subscribe to SDK events

Once Onirix AR Engine SDK has been successfully initialized, you will have to subscribe to events in order to know when an Surface, Image or QR code has been detected and retrieve its AR pose to update the 3D renderer camera accordingly.

There are 4 events you may want to subscribe depending on the tracking mode:

| Event | Description | | ------ | ----------- | | OnirixSDK.Events.OnDetected | This event will be called when an Image, QR code or Space is detected, returning an identifier. The identifier corresponds to the Image or Space scene OID in Onirix Studio or the QR Code decoded content in case of a QR code. This event won't never be called in Surface mode. | | OnirixSDK.Events.OnLost | This event will be called when an already detected Image or QR code is lost, returning an identifier. The identifier corresponds to the Image scene OID in Onirix Studio or the QR Code decoded content in case of a QR code. This event won't never be called in Surface mode. | | OnirixSDK.Events.OnPose | This event will be called every time a new object pose is computed. Returned pose is a column major view-model matrix in OpenGL coordinates that you must forward to your 3D renderer. | | OnirixSDK.Events.OnResize | This event will be called every time your device switches between landscape and portrait orientation. When that happens, you must obtain new camera parameters through getCameraParameters function of the SDK and update your renderer with them. | | OnirixSDK.Events.OnTouch | This event will be called every time users tap the screen (touch start and end), returning the clip-space (between -1 and 1) screen position of the touch. You may wan't to use it for raycasting and interacting with the scene when users touch. | | OnirixSDK.Events.OnTouchStart | Similar to OnTouch, but called when the touch starts. | | OnirixSDK.Events.OnTouchEnd | Similar to OnTouch, but called when the touch ends. | | OnirixSDK.Events.OnHitTestResult | This event will be called every time a surface is detected throwing a ray forward from camera origin (screen center). It is useful to display a placeholder of a surface before placing the 3D scene. It only works with surface tracking mode. | | OnirixSDK.Events.OnFrame | This event will be called every frame. Useful for synchronizing tracking and 3D rendering. | | OnirixSDK.Events.SessionEnd | Called when user exits the AR experience (only Surface and Spatial modes when using WebXR). |

This is how you should subscribe to events:

OX.subscribe(OnirixSDK.Events.OnDetected, function (id) {
   renderer.load3DScene(id);
});

OX.subscribe(OnirixSDK.Events.OnPose, function (pose) {
   renderer.updateCameraPose(pose);
});

OX.subscribe(OnirixSDK.Events.OnLost, function (id) {
   renderer.unload3DScene(id);
});

OX.subscribe(OnirixSDK.Events.OnResize, function () {
   renderer.updateCameraParams();
});

OX.subscribe(OnirixSDK.Events.OnTouch, function (touchPos) {
   renderer.onTouch(touchPos);
});

OX.subscribe(OnirixSDK.Events.OnHitTestResult, function (hitResult) {
   renderer.onHitTestResult(hitResult);
});

OX.subscribe(OnirixSDK.Events.OnFrame, function () {
   renderer.render();
});

Start tracking (and other lifecycle methods)

Once initialization is complete (by calling init), you must call the start method in order for the system to start tracking.

While using Surface mode, it is important to call this method once you've selected the surface to place the model via OnHitTestResult subscription for better tracking results.

In addition to start, there is also a stop method intented for stopping tracking at any time and stop receiving pose updates.

If your app requires to switch between several tracking modes, you can release the SDK by calling the destroy method and calling init again with another tracking configuration.

Coordinate system

Onirix uses a right handed coordinate system with X running horizontally left to right, Y running vertically bottom to top, and positive Z coming out of the screen:

This coordinate system was chosen to match WebGL convention that is also used by many 3D libraries like Three.js. However, there are others like Babylon.js may need some conversions to get the same results. We encourage you to visit our samples to see how we deal with these coordinate transforms for each 3D engine.

Surface mode

While in surface mode, -Z points the horizon, Y points up aligned with gravity, and X is the cross vector between both.

Image mode

While in image mode, positive Y is orthogonal coming out from marker image, X goes right and Z down (-Y in 2D marker coordinates).

Curved mode

While in curved mode, -Z is orthogonal coming out from marker image, X goes right and Y down.

Spatial mode

While in Spatial mode, the coordinate system aligns with the scanned environment. This means that, similar to Surface mode, the -Z axis points towards the horizon, the Y axis points upward, aligned with gravity, and the X axis is perpendicular to both, forming the cross vector. However, in Spatial mode, these axes are rotated in such a way that they match the orientation and position of the previously scanned space, ensuring that virtual objects are accurately placed within the real-world context.

Sample projects

You can visit our GitHub account for samples using different tracking modes and 3D rendering libraries:

https://github.com/onirix-ar/webar-sdk-samples

Local development and testing

Onirix AR Engine SDK usage is restricted to purchased domains, however, the following domains and addresses are whitelisted for development:

localhost
127.0.0.1
192.168.*
10.50.*
10.10.*
*.ngrok.io

In order to test the SDK from these local addresses, it is mandatory you serve your files through HTTPs. There are plenty of tools out there that may help you with this by generating an self-signed certificate and serving your files with a single command.

Take a look at http-server utility.

Compatibility and browser support

Onirix AR Engine SDK is compatible with the following browsers:

|OS|Version|Supported browsers| |----------|--------------|----------| |iOS|iOS 11+|Safari, Safari web views (iOS 14.3+)| |Android|Any|Chrome, Firefox, Samsung Internet, Edge, native Android web views (Android 4.4+) *|

*Other browsers may be also compatible if supporting the following capabilities: WebGL, CameraAPI, DeviceMotionAPI, WebAssembly (WASM).

Devices must also conform to the following hardware requirements:

Rear camera with at least VGA resolution (640x480px)
Gyroscope sensor (only required for Surface and Spatial modes).
2 GB of RAM

Licensing

Using Onirix AR Engine SDK is limited to Onirix users.

There is no limit to the number of Web AR experiences you create with the SDK. However, they could be only consumed a limited number of times, regulated by your plan's monthly views. A view is generated every time a user opens your experience and it loads without errors.

Self-hosting the experience in a public site (not local development) requires to purchase a domain (often a domain is included with certain Onirix plans).

See Onirix Pricing for more information.