node-red-contrib-facial-recognition

Node-RED - Provides a node-red node for Facial Detection & Facial Recognition.

Table of Contents

• Requirements
• Install
• About
• Usage
• Node_Properties
  • Name
  • Image
  • Settings
  • Bindings
  • FaceDetector
  • FaceRecognition
• Example_Flows
• Simple_Queue_Method
• Heavy_image_processing_or_mjpeg_video_stream
• FaceRecognition_Persistant_labeledFaceDescriptors
• Bugs_Feature_request
• License
• Work
• Contributor

Requirements

Node version 12.x or higher. Use node -v in terminal to find out

Node-red version 1.x or higher.

Install

Install with node-red Palette Manager or,

Run the following command in your Node-RED user directory - typically ~/.node-red:

npm install node-red-contrib-facial-recognition

Windows users: If your having issues and use the trouble shooting guide. Run the commands in the Windows troubleshooting guide from within your Node-RED user directory - typically ~/.node-red

Docker users: Node-red's official docker will not work. It is based on alpine and is not libc6 compatible. Please see this thread for how to install a docker version of node-red that works. https://discourse.nodered.org/t/announce-node-red-contrib-facial-recognition/37384/53

About

At its core it uses @vladmandic/face-api and @tensorflow/tfjs-node ver.2.7.x and it can use @tensorflow/tfjs-node-gpu for the CUDA crazy amongst you.

vladmandic was a big help for us nodejs guys. After finding a bug and fielding questions, he took the time to make a nodejs build specific to tfjs-node. If you like this node-red-contrib-facial-recognition, I highly recommend you take the time to goto vladmandic's github page https://github.com/vladmandic/face-api and throw his repo a star.

Usage

Takes a buffered image and runs TensorFlow Facial Detection and/or Facial Recognition to Detect:

• Faces in an image
• Facial landmarks
• Compute Face Descriptors
• Face Expressions
• Age & Gender Recognition
• Face Recognition by Matching Descriptors

Node_Properties

Name

Define the msg name if you wish to change the name displayed on the node.

Image

You can change the msg property value that you send a buffered image of your choice to.Example: msg.NameOfYourChoice

Image size and the avalible memory on the device your using. Sending a 4mb image on a raspberry pi will crash the node. If your running other stuff this size will be lower. The tensorfolow core only sends out a Standard Error when you push the limits of your devices memory. If your unsure look at your node-red console log to see if you unsure if the image size is too big.

Settings

This is optional, you do not have to send it anything. Used to override settings in the nodes config Properties menu.

You can change the msg property value that you send an object of your choice to. Example: msg.NameOfYourChoice

Sending a object to this msg property value will override any settings in the nodes config Properties menu. Great for using input messages to change settings on the fly. Please see/inport the example Flow section of this documentation for better understanding. Example:

msg.settings = {
  FaceDetector :
  {
    SsdMobilenetv1 :
    {
      maxResults : 4,
      minConfidence : 0.6
    }
  },
  Tasks :
  {
    detectAllFaces :
    {
      withFaceLandmarks : true,
      withFaceExpressions : true,
      withAgeAndGender : true,
      withFaceDescriptors : true
    }
  },
  FaceRecognition :
  {
    enabled :
    {
      KnownFacesPath : "/example/known_face",
      distanceThreshold : 0.6,
      ReInitializeFaceMatcher : false
    }
  }            
};
return msg;

You do not have to fill out every option. You can omit any object key and its value. This node will then use the settings found in the nodes config Properties menu for that omitted object key and its value.

Note: ReInitializeFaceMatcher Set this value to true if you have changed/edited/added images or image folders to your KnownFacesPath to ReInitialize the FaceMatcher. Used to process all the images into Labeled Face Descriptors for each dir name and individual descriptions for images. Do not leave set to true! it takes significant time to process. Once its ran after you have made changes to images or image folder it is saved to context and used for Facial Recognition.

Else you can just re-deploy node-red and ReInitializeFaceMatcher will run one time only on the first image you send.

Bindings

By default it is set to CPU - @tensorflow/tfjs-node, this will use your CPU to process images. However you may choose to install @tensorflow/tfjs-node-gpu to utilize your video card to process images. This is not an easy process to get CUDA working. However if you go down this rabbit hole the benefits in time to process images are significant.

Good luck.

FaceDetector

SsdMobilenetv1 - A Single Shot Multibox Detector; based on MobileNetV1. Computes the locations of each face in an image and returns the bounding boxes with it's probability for each face. High accuracy in detecting face bounding boxes at the cost of time to compute.

• maxResults - The max number of faces to return
• minConfidence - returns results for face(s) in a image above Confidence threshold

tinyFaceDetector - a fast realtime face detector, and less resource consuming compared to the SSD Mobilenet V1 face detector. It is poor at detecting small faces. Best face detector on resource limited devices.

• inputSize - size at which image is processed, the smaller the faster, number must be divisible by 32. Common sizes are 128, 160, 224, 320, 416, 512, 608
• scoreThreshold - returns results for face(s) in a image above Confidence threshold

Tasks

detectAllFaces - Returns an array of objects. Utilize the selected FaceDetector to detect multiple faces in a buffered image sent in message by user detectSingleFace - Returns an object. Utilize the selected FaceDetector to detect a single face in a buffered image sent in message by user. If image contains multiple faces it will only detect one of them, hopefully the highest probability one.

• withFaceLandmarks - computes landmarks for each detected face(s)
• withFaceExpressions - recognize face expressions of each face(s)
• withAgeAndGender - estimate age and recognize gender of each face(s)
• withFaceDescriptor - computes the face descriptors for each face(s)

FaceRecognition

disabled - Don't use any facial recognition enabled - Performs face recognition, by comparing reference face descriptor(s) to determine the similarity to query face descriptor(s). Note: FaceDetector minConfidence Properties affect the labeledFaceDescriptors. If you have a minConfidence of .9 you may miss a bunch of faces when building your labeledFaceDescriptors. after you run the node once or supply it a labeledFaceDescriptors from a persistant storage you can set the value to any level you wish for screening the input image you send

• KnownFacesPath - The location of the main folder that contains all subfolders, labeled with persons name. If you change the value of this path you mush use a full path. Example: /home/myuser/name/TheFolderIPutAllTheLabledFaceFolderIn. Subfolders should contain close-up face images of the individual. The actual name of the file in this folder does not matter. Please look at the structure of the example folder for more understanding. The name of the subfolder is what is used to label the faces for facial recognition. Note: If you have changed/edited/added images or image folders to your KnownFacesPath you must redeploy the node to ReInitialize the FaceMatcher. On the first image you send, FaceMatcher is ran to process all the images into Labeled Face Descriptors for each dir name and individual descriptions for images. Then it is saved to context and used for Facial Recognition for all future images you send in a message.
• distanceThreshold - returns results based on measures of how far away, Euclidean distance of face descriptor, the user submitted image is compared to how far away, Euclidean distance of face descriptors, it is to all the faces found in the labeled subfolder that are above the distanceThreshold. Simply put: return names for person if it is this distanceThreshold of confidence. The lower the distanceThreshold is the more likely you are to get a incorrect match. The higher the distanceThreshold is the more likely it is that a person will not be recognized.

Example_Flows

BASIC:

Click Me For BASIC example

Advanced:

NOTE: other node-red nodes required

node-red-node-annotate-image

node-red-contrib-image-output

Click Me For Advanced example

Simple_Queue_Method

If your having issues with overloading the node with too many images that tensorflow can't handle you can always add a Queue

This will take up ram as the queue grows and tensorflow is ram hungry. Only use this method if the queue will decrease in time. Don't let it grow out of control.

Example Flow:

NOTE: other node-red nodes required

node-red-contrib-simple-message-queue

Click Me For Simple_Queue_Method example

Heavy_image_processing_or_mjpeg_video_stream

Every output object message from this node has sec_to_complete with the amount of time it took to process the image. Based on your device/processing speed/CPU or CUDA will vary the amount of time it takes to process the image.

So if it takes 0.623 seconds to do a facialrecognition and your sending it 15 frames a second you will create a backlog of work and overflow the node.

The best thing to do is use multiple facial-recognition nodes to process the images as individual workers.

then check to see if it is keeping up.

Example Flow:

NOTE: other node-red nodes required

node-red-contrib-msg-router

Click Me For Heavy_image_processing_or_mjpeg_video_stream example

FaceRecognition_Persistant_labeledFaceDescriptors

Note: FaceDetector minConfidence Properties affect the labeledFaceDescriptors. If you have a minConfidence of .9 you may miss a bunch of faces when building your labeledFaceDescriptors. after you run the node once or supply it a labeledFaceDescriptors from a persistant storage you can set the value to any level you wish for screening the input image you send.

You can save the FaceRecognition labeledFaceDescriptors to persistant storage solution so you don't have to load all the images in your KnownFacesPath folder every time you deploy node red or if node-red/your device restarts.

This could be a HUGE time savings if you have thousands of people in your KnownFacesPath folder.

Also this shows how the msg.settings is used. Just un-comment anything you wish to override in the Properties menu of the node.

Example Flow:

Click Me For FaceRecognition_Persistant_labeledFaceDescriptors example

Bugs_Feature_request

Please report bugs and feel free to ask for new features directly on GitHub.

License

This project is licensed under Apache 2.0 license.

All photos are free to use and provided by: unsplash.

Work

_Need a node? _Need automation work? _Need computers to flip switches?

Contact me at [email protected]

Contributor

Thanks to:

The @tensorflow/tfjs-node team for supporting and maintaining a repo that allows us JS guys to create cool stuff.

vladmandic and his @vladmandic/face-api for help and support in creating a nodeJS specific build for the face-api.

protocolus for his work on finding images for the user example.

Joshua Rondeau for free use of his photos.

Brandon Atchison for free use of his photos.

zenofmud for bug fixes and new feature.

M0ebiu5 for helping change the logic of KnownFacesPath

Heavy_image_processing_or_mjpeg_video_stream

release notes

0.0.0 = (majorchange) . (new_feature) . (bugfix-simple_mod) version 0.30.105 - new_feature - Updated to use tfjs-node 3.7.0 or higher, face-api 1.3.1 or higher, fixes the agent requierment of 6.x or higher version 0.29.105 - documentation - Added queue documentation, added documentaion for array / object version 0.29.104 - new_feature - Checks for node version greather than 12, added checks for @vladmandic/face-api version 0.28.104 - documentation - Added Docker use info and Requirements section version 0.28.103 - documentation - Switched to using msg-router for Heavy_image_processing_or_mjpeg_video_stream version 0.28.102 - bug - fix how README.md is displayed on web pages version 0.28.101 - documentation - Update FaceRecognition_Persistant_labeledFaceDescriptors to version based on zenofmud version 0.28.100 - bug - M0ebiu5 change the logic of KnownFacesPath version 0.28.99 - bug - fix faceapi = require for gpu, so the gpu option will work and up date documentation for path version 0.28.98 - bug - docker post install script version 0.28.97 - documentation - fix missing'<' for details version 0.28.96 - documentation - add expand for examples version 0.28.95 - documentation - correction version 0.28.94 - documentation - Windows troubleshooting guide version 0.28.93 - bug - fix disc in JSON file version 0.28.92 - new_feature - persistent storage of labeledFaceDescriptors version 0.27.92 - bug - remove github code mistake version 0.27.91 - new_feature - zenofmud submitted a fix for MacOS .DS_Store file removal from files version 0.26.91 - bug/new_feature - zenofmud submitted fix for MacOS .DS_Store file removal from folders, and a spelling error version 0.26.90 - bug - file location fix version 0.26.89 - new_feature - Removed eval and other optimizations version 0.25.89 - bug - UnhandledPromiseRejection - added catch to handle these version 0.25.88 - new_feature -added error checks for user submiting bad/unkown images to the folder or as msg.input version 0.24.88 - bug -typo fix in documentation version 0.24.87 - documentation - added Heavy_image_processing_or_mjpeg_video_stream section and example for it version 0.24.86 - documentation - Fix documentation, added examples and intro image to top of page. version 0.24.85 - documentation - Fix documentation, updating examples! version 0.24.84 - First Public release