@jigra-mlkit/face-mesh-detection
v5.4.0
Published
Jigra plugin for ML Kit Face Mesh Detection.
Downloads
2
Readme
@jigra-mlkit/face-mesh-detection
Unofficial Jigra plugin for ML Kit Face Mesh Detection.[^1]
Installation
npm install @jigra-mlkit/face-mesh-detection
npx jig sync
Android
Variables
This plugin will use the following project variables (defined in your app’s variables.gradle
file):
$mlkitFaceMeshDetectionVersion
version ofcom.google.mlkit:face-mesh-detection
(default:16.0.0-beta1
)
Configuration
No configuration required for this plugin.
Usage
import { FaceMeshDetection, UseCase } from '@jigra-mlkit/face-mesh-detection';
const processImage = async () => {
const { faceMeshs } = await FaceMeshDetection.processImage({
path: 'path/to/image.jpg',
useCase: UseCase.FaceMesh,
});
return faceMeshs;
};
API
processImage(...)
processImage(options: ProcessImageOptions) => Promise<ProcessImageResult>
Detects face mesh from the supplied image.
Only available on Android.
| Param | Type |
| ------------- | ------------------------------------------------------------------- |
| options
| ProcessImageOptions |
Returns: Promise<ProcessImageResult>
Since: 5.3.0
Interfaces
ProcessImageResult
| Prop | Type | Description | Since |
| --------------- | ----------------------- | ------------------------ | ----- |
| faceMeshs
| FaceMesh[] | The detected face meshs. | 5.3.0 |
FaceMesh
Represents a face mesh detected by FaceMeshDetector
.
When BoundingBoxOnly
is selected, FaceMesh
only contains valid bounding box.
When FaceMesh
is selected, FaceMesh
also contains a group of 468 3D face mesh points and related triangle information.
Each point is represented by FaceMeshPoint
describing a specific position in detected face.
The triangle information is a group of 3 FaceMeshPoint
s representing a valid surface on Face (e.g. a valid small surface on nose tip).
| Prop | Type | Description | Since |
| -------------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- |
| bounds
| Rect | Returns the axis-aligned bounding rectangle of the detected face mesh. | 5.3.0 |
| contours
| Contours | Returns contours with a list of FaceMeshPoint
representing the detected face. | 5.3.0 |
| faceMeshPoints
| FaceMeshPoint[] | Returns a list of FaceMeshPoint
representing the whole detected face. | 5.3.0 |
| triangles
| Triangle[] | Returns a list of Triangle
representing logical triangle surfaces of detected face. Each Triangle
contains 3 FaceMeshPoint
, representing 3 points of the triangle surface. The sequence of the 3 points are constant and always counter clockwise in face mesh. | 5.3.0 |
Rect
Rect holds four integer coordinates for a rectangle.
| Prop | Type | Description | Since |
| ------------ | ------------------- | ---------------------------------------------------- | ----- |
| left
| number | The X coordinate of the left side of the rectangle. | 5.3.0 |
| top
| number | The Y coordinate of the top of the rectangle. | 5.3.0 |
| right
| number | The X coordinate of the right side of the rectangle. | 5.3.0 |
| bottom
| number | The Y coordinate of the bottom of the rectangle. | 5.3.0 |
Contours
Represents contours with their face mesh points.
| Prop | Type | Description | Since |
| ------------------------ | ---------------------------- | -------------------------------------------------------- | ----- |
| faceOval
| FaceMeshPoint[] | Returns all points for the FaceOval
contour. | 5.3.0 |
| leftEyebrowTop
| FaceMeshPoint[] | Returns all points for the LeftEyebrowTop
contour. | 5.3.0 |
| leftEyebrowBottom
| FaceMeshPoint[] | Returns all points for the LeftEyebrowBottom
contour. | 5.3.0 |
| rightEyebrowTop
| FaceMeshPoint[] | Returns all points for the RightEyebrowTop
contour. | 5.3.0 |
| rightEyebrowBottom
| FaceMeshPoint[] | Returns all points for the RightEyebrowBottom
contour. | 5.3.0 |
| leftEye
| FaceMeshPoint[] | Returns all points for the LeftEye
contour. | 5.3.0 |
| rightEye
| FaceMeshPoint[] | Returns all points for the RightEye
contour. | 5.3.0 |
| upperLipTop
| FaceMeshPoint[] | Returns all points for the UpperLipTop
contour. | 5.3.0 |
| upperLipBottom
| FaceMeshPoint[] | Returns all points for the UpperLipBottom
contour. | 5.3.0 |
| lowerLipTop
| FaceMeshPoint[] | Returns all points for the LowerLipTop
contour. | 5.3.0 |
| lowerLipBottom
| FaceMeshPoint[] | Returns all points for the LowerLipBottom
contour. | 5.3.0 |
| noseBridge
| FaceMeshPoint[] | Returns all points for the NoseBridge
contour. | 5.3.0 |
FaceMeshPoint
Represents a 3D point in face mesh.
The index is an unique ID meaning a fixed position on face, ranging from 0 to 467.
In Point3D
, x
and y
are pixel location of detected face in InputImage
.
z
is also scaled to image size, while the origin will be somewhere in the center of all 468 face mesh points.
| Prop | Type | Description | Since |
| ----------- | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- |
| index
| number | Gets the index of the face mesh point, ranging from 0 to 467. For each specific point, the index is a constant value. | 5.3.0 |
| point
| Point3D | Gets a 3D point in face mesh. Inside Point3D
, X
and Y
means a 2D position in original image. More information on the Z
value: - The unit of measure for the Z
value is the same as X
and Y
. - The smaller the Z
value, the closer that landmark is to the camera. - The Z
origin is approximately at the center of all 468 face mesh points. Z
value will be negative if the point is close to camera and will be positive if the point is away from the camera. | 5.3.0 |
Point3D
Represents a 3D point.
| Prop | Type | Description | Since |
| ------- | ------------------- | --------------------------------- | ----- |
| x
| number | Returns the X value of the point. | 5.3.0 |
| y
| number | Returns the Y value of the point. | 5.3.0 |
| z
| number | Returns the Z value of the point. | 5.3.0 |
Triangle
Represents a triangle with 3 generic points.
| Prop | Type | Description | Since |
| ------------ | ---------------------------- | ----------------------------------------------------------------- | ----- |
| points
| FaceMeshPoint[] | Returns all points inside the Triangle
. | 5.3.0 |
ProcessImageOptions
| Prop | Type | Description | Default | Since |
| ------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------- | ----- |
| path
| string | The local path to the image file. | | 5.3.0 |
| useCase
| UseCase | Sets the use case. When BoundingBoxOnly
is selected, the returned FaceMesh
only contains bounding box. When FaceMesh
is selected, the returned FaceMesh
contains bounding box as well as 468 FaceMeshPoint
and triangle information. It detects at most 2 faces in this case and it is slower than BoundingBoxOnly
. | UseCase.FaceMesh | 5.3.0 |
Enums
UseCase
| Members | Value | Description | Since |
| --------------------- | -------------- | ------------------------------------------------------------------------------------- | ----- |
| BoundingBoxOnly
| 0 | Return bounding box for detected face. | 5.3.0 |
| FaceMesh
| 1 | Return face mesh info for detected face. It detects at most 2 faces in this use case. | 5.3.0 |
Terms & Privacy
This plugin uses the Google ML Kit:
License
See LICENSE.
[^1]: This project is not affiliated with, endorsed by, sponsored by, or approved by Google LLC or any of their affiliates or subsidiaries.