facenet-wehaus-dist
v0.10.7
Published
Solve face verification, recognition and clustering problems: a TensorFlow backed FaceNet implementation for Node.js.
Downloads
11
Maintainers
Readme
FACENET
A TensorFlow backed FaceNet implementation for Node.js, which can solve face verification, recognition and clustering problems.
FaceNet is a deep convolutional network designed by Google, trained to solve face verification, recognition and clustering problem with efficiently at scale.
- directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity.
- optimize the embedding face recognition performance using only 128-bytes per face.
- achieves accuracy of 99.63% on Labeled Faces in the Wild (LFW) dataset, and 95.12% on YouTube Faces DB.
INSTALL
$ npm install facenet numjs flash-store
Peer Dependencies
numjs
flash-store
EXAMPLE
The follow examples will give you some intuitions for using the code.
- demo exmaple will show you how to do
align
for face alignment andembedding
to get face feature vector. - visualize example will calculate the similarity between faces and draw them on the photo.
1. Demo for API Usage
TL;DR: Talk is cheap, show me the code!
import { Facenet } from 'facenet'
const facenet = new Facenet()
// Do Face Alignment, return faces
const imageFile = `${__dirname}/../tests/fixtures/two-faces.jpg`
const faceList = await facenet.align(imageFile)
for (const face of faceList) {
console.log('bounding box:', face.boundingBox)
console.log('landmarks:', face.facialLandmark)
// Calculate Face Embedding, return feature vector
const embedding = await facenet.embedding(face)
console.log('embedding:', embedding)
}
faceList[0].embedding = await facenet.embedding(faceList[0])
faceList[1].embedding = await facenet.embedding(faceList[1])
console.log('distance between the different face: ', faceList[0].distance(faceList[1]))
console.log('distance between the same face: ', faceList[0].distance(faceList[0]))
Full source code can be found at here: https://github.com/huan/node-facenet/blob/master/examples/demo.ts
The output should be something like:
image file: /home/zixia/git/facenet/examples/../tests/fixtures/two-faces.jpg
face file: 1-1.jpg
bounding box: {
p1: { x: 360, y: 95 },
p2: { x: 589, y: 324 }
}
landmarks: {
leftEye: { x: 441, y: 181 },
rightEye: { x: 515, y: 208 },
nose: { x: 459, y: 239 },
leftMouthCorner: { x: 417, y: 262 },
rightMouthCorner: { x: 482, y: 285 }
}
embedding: array([ 0.02453, 0.03973, 0.05397, ..., 0.10603, 0.15305,-0.07288])
face file: 1-2.jpg
bounding box: {
p1: { x: 142, y: 87 },
p2: { x: 395, y: 340 }
}
landmarks: {
leftEye: { x: 230, y: 186 },
rightEye: { x: 316, y: 197 },
nose: { x: 269, y: 257 },
leftMouthCorner: { x: 223, y: 273 },
rightMouthCorner: { x: 303, y: 281 }
}
embedding: array([ 0.03241, -0.0737, 0.0475, ..., 0.07235, 0.12581,-0.00817])
2. Visualize for Intuition
- Face is in the green rectangle.
- Similarity(distance) between faces showed as a number in the middle of the line.
- To identify if two faces belong to the same person, we could use an experiential threshold of distance: 0.75.
$ git clone [email protected]:zixia/node-facenet.git
$ cd facenet
$ npm install
$ npm run example:visualize
01:15:43 INFO CLI Visualized image saved to: facenet-visulized.jpg
3. Get the diffence of two face
Get the two face's distance, the smaller the number is, the similar of the two face
import { Facenet } from 'facenet'
const facenet = new Facenet()
const imageFile = `${__dirname}/../tests/fixtures/two-faces.jpg`
const faceList = await facenet.align(imageFile)
faceList[0].embedding = await facenet.embedding(faceList[0])
faceList[1].embedding = await facenet.embedding(faceList[1])
console.log('distance between the different face: ', faceList[0].distance(faceList[1]))
console.log('distance between the same face: ', faceList[0].distance(faceList[0]))
Output:
distance between the different face: 1.2971515811057608
distance between the same face: 0
In the example,
faceList[0] is totally the same with faceList[0], so the number is 0
faceList[1] is different with faceList[1], so the number is big.
If the number is smaller than 0.75, maybe they are the same person.
Full source code can be found at here: https://github.com/huan/node-facenet/blob/master/examples/distance.ts
4. Save the face picture from a picture
Recognize the face and save the face to local file.
import { Facenet } from 'facenet'
const facenet = new Facenet()
const imageFile = `${__dirname}/../tests/fixtures/two-faces.jpg`
const faceList = await facenet.align(imageFile)
for (const face of faceList) {
await face.save(face.md5 + '.jpg')
console.log(`save face ${face.md5} successfuly`)
}
console.log(`Save ${faceList.length} faces from the imageFile`)
Full source code can be found at here: https://github.com/huan/node-facenet/blob/master/examples/get-face.ts
FACENET MANAGER
UNDER HEAVY DEVELOPMENT NOW
Roadmap: release facenet-manager on version 0.8
The above ascii recording is just for demo purpose. Will replace it with facenet-manager later.
DOCUMENT
INSTALL & REQUIREMENT
$ npm install facenet
OS
Supported:
- [x] Linux
- [x] Mac
- [ ] Windows
Dependency
- Node.js >= 7 (8 is recommend)
- Tensorflow >= 1.2
- Python3 >=3.5 (3.6 is recommend)
Make sure you run those commands under Ubuntu 17.04:
sudo apt install python3-pip
pip3 install setuptools --upgrade
Ram
| Neural Network Model | Task | Ram | | --- | --- | --- | | MTCNN | Facenet#align() | 100MB | | Facenet | Facenet#embedding() | 2GB |
If you are dealing with very large images(like 3000x3000 pixels), there will need additional 1GB of memory.
So I believe that Facenet will need at least 2GB memory, and >=4GB is recommended.
API
Neural Network alone is not enough. It's Neural Network married with pre-trained model, married with easy to use APIs, that yield us the result that makes our APP sing.
Facenet is designed for bring the state-of-art neural network with bleeding-edge technology to full stack developers.
Facenet
import { Facenet } from 'facenet'
const facenet = new Facenet()
facenet.quit()
1. Facenet#align(filename: string): Promise<Face[]>
Do face alignment for the image, return a list of faces.
2. Facenet#embedding(face: Face): Promise<FaceEmbedding>
Get the embedding for a face.
face.embedding = await facenet.embedding(face)
Face
Get the 128 dim embedding vector for this face.(After alignment)
import { Face } from 'facenet'
console.log('bounding box:', face.boundingBox)
console.log('landmarks:', face.facialLandmark)
console.log('embedding:', face.embedding)
ENVIRONMENT VARIABLES
FACENET_MODEL
FaceNet neural network model files, set to other version of model as you like.
Default is set to models/
directory inside project directory. The pre-trained models is come from 20170512-110547, 0.992, MS-Celeb-1M, Inception ResNet v1, which will be download & save automatically by postinstall
script.
$ pwd
/home/zixia/git/node-facenet
$ ls models/
20170512-110547.pb
model-20170512-110547.ckpt-250000.index
model-20170512-110547.ckpt-250000.data-00000-of-00001
model-20170512-110547.meta
DOCKER
DEVELOP
$ git clone [email protected]:zixia/node-facenet.git
$ cd facenet
$ npm install
$ npm test
COMMAND LINE INTERFACES
align
Draw a rectangle with five landmarks on all faces in the input_image, save it to output_image.
$ ./node_modules/.bin/ts-node bin/align.ts input_image output_image
embedding
Output the 128 dim embedding vector of the face image.
$ ./node_modules/.bin/ts-node bin/embedding.ts face_image
RESOURCES
Machine Learning
- Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning
- Face recognition using Tensorflow
- Google: Our new system for recognizing faces is the best one ever
- A tensorflow implementation of "Deep Convolutional Generative Adversarial Networks
- What does Locality Sensitive Hashing Forests do? · maheshakya/my_first_project Wiki
- Average Face : OpenCV ( C++ / Python ) Tutorial
Python3
- Google Python Style Guide
- PyLint, PyChecker or PyFlakes?
- Useful Python Modules: Flake8
- PEP 8 - Style Guide for Python Code
- Python 3.6 venv — Creation of virtual environments
1. Typing
- Mypy syntax cheat sheet (Python 3)
- Python 3 Type Hints and Static Analysis
- typing — Support for type hints
1. NumJS
- Stackoverflow: numpy-like package for node
- Read/manipulate/display images using NumJs
- Numjs - Like NumPy, in JavaScript
- ndarray - Modular multidimensional arrays for JavaScript
Dataset
TODO
- [x] NPM Module:
facenet
- [x] Docker Image:
zixia/facenet
- [ ] Examples
- [x] API Usage Demo
- [x] Triple Distance Visulization Demo
- [ ] Performance Test(Align/Embedding/Batch)
- [ ] Validation Test(LFW Accuracy)
- [ ] Neural Network Models
- [x] Facenet
- [x] Mtcnn
- [ ] Batch Support
- [ ] ~~Python3
async
&await
~~ - [ ] Divide Different Neural Network to seprate class files(e.g. Facenet/Mtcnn)
- [x] K(?)NN Alghorithm Chinese Whispers
- [ ] TensorFlow Sereving
- [ ] OpenAPI Specification(Swagger)
INSPIRATION
This repository is heavily inspired by the following implementations:
- FaceNet by David Sandberg @davidsandberg
- OpenFace by CMU Satya Lab @cmusatyalab
CREDITS
- Face alignment using MTCNN: Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
- Face embedding using FaceNet: FaceNet: A Unified Embedding for Face Recognition and Clustering
- TensorFlow implementation of the face recognizer: Face recognition using Tensorflow
CONTRIBUTE
FaceNet Badge
[![Powered by FaceNet](https://img.shields.io/badge/Powered%20By-FaceNet-green.svg)](https://github.com/huan/node-facenet)
CHANGELOG
v0.9 master unstable
v0.8 (Apr 2018)
- Added
facenet-manager
command line tool for demo/validate/sort photos - Switch to
FlashStore
npm module as key-value database
v0.3 Sep 2017
- Added three cache classes: AlignmentCache & EmbeddingCache & FaceCache.
- Added cache manager utilities: embedding-cache-manager & alignment-cache-manager & face-cache-manager
- Added Dataset manager utility: lfw-manager (should be dataset-manager in future)
- BREAKING CHANGE:
Face
class refactoring.
v0.2 Aug 2017 (BREAKING CHANGES)
Facenet#align()
now accept a filename string as parameter.- BREAKING CHANGE:
FaceImage
class had been removed. - BREAKING CHANGE:
Face
class refactoring.
v0.1 Jul 2017
npm run demo
to visuliaze the face alignment and distance(embedding) in a three people photo.- Facenet.align() to do face alignment
- Facenet.embedding() to calculate the 128 dim feature vector of face
- Initial workable version
TROUBLESHOOTING
Dependencies
OS | Command
----- | -----
os x | brew install pkg-config cairo pango libpng jpeg giflib
ubuntu | sudo apt-get install libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev build-essential g++
fedora | sudo yum install cairo cairo-devel cairomm-devel libjpeg-turbo-devel pango pango-devel pangomm pangomm-devel giflib-devel
solaris | pkgin install cairo pango pkg-config xproto renderproto kbproto xextproto
windows | instructions on our wiki
more os see node-canvas Wiki.
FAQ
facenet-manager
display not right under Windows
See: Running Terminal Dashboards on Windows
- Error when install:
No package 'XXX' found
It's related with the NPM module canvas
.
Error messages:
No package 'pixman-1' found
No package 'cairo' found
No package 'pangocairo' found
Solution for Ubuntu 17.04:
sudo apt install -y libpixman-1-dev
sudo apt-get install -y libcairo2-dev
sudo apt-get install -y libpango1.0-dev
Solution for Mac:
brew install python3
brew install pkg-config
brew install cairo
brew install pango
brew install libpng
brew install libjpeg
- Error when install:
fatal error: jpeglib.h: No such file or directory
It's related with the NPM module canvas
.
Solution for Ubuntu 17.04:
sudo apt-get install -y libjpeg-dev
- Error when run:
Error: error while reading from input stream
It is related with the libjpeg
package
Solution for Mac:
brew install libjpeg
- Error when run:
Error: Cannot find module '../build/Release/canvas.node'
at Function.Module._resolveFilename (module.js:527:15)
at Function.Module._load (module.js:476:23)
at Module.require (module.js:568:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/Users/jiaruili/git/node-facenet/node_modules/canvas/lib/bindings.js:3:18)
at Module._compile (module.js:624:30)
at Object.Module._extensions..js (module.js:635:10)
at Module.load (module.js:545:32)
at tryModuleLoad (module.js:508:12)
at Function.Module._load (module.js:500:3)
It seems the package not installed in a right way, like sharp
, canvas
, remove the package and reinstall it.
run
rm -rf node node_modules/canvas
// if sharp, then remove sharp folder
npm install
- Error when install
> [email protected] postinstall:models /Users/jiaruili/git/rui/node-facenet
> set -e && if [ ! -d models ]; then mkdir models; fi && cd models && if [ ! -f model.tar.bz2 ]; then curl --location --output model.tar.bz2.tmp https://github.com/huan/node-facenet/releases/download/v0.1.9/model-20170512.tar.bz2; mv model.tar.bz2.tmp model.tar.bz2; fi && tar jxvf model.tar.bz2 && cd -
x 20170512-110547.pb
x model-20170512-110547.ckpt-250000.data-00000-of-00001: (Empty error message)
tar: Error exit delayed from previous errors.
It seems this because not get the full model file successfully. See #issue63
Solution:
download the file from https://github.com/huan/node-facenet/releases/download/v0.1.9/model-20170512.tar.bz2
rename the file model.tar.bz2
and move it to the folder models
try npm install
again
SEE ALSO
- Face Blinder: Assitant Bot for Whom is Suffering form Face Blindess
- Wechaty Blinder: Face Blinder Bot Powered by Wechaty
AUTHOR
Huan LI <[email protected]> (http://linkedin.com/in/zixia)
COPYRIGHT & LICENSE
- Code & Docs © 2017 Huan LI <[email protected]>
- Code released under the Apache-2.0 License
- Docs released under Creative Commons