Three lines of code to implement a computer vision application using Tensorflow.js

Let’s see an easy way to implement a computer vision application using tensorflow.js!!!

Introduction

What is TensorFlow.js?

TensorFlow.js is an open-source library for machine learning application in JavaScript and Develop ML models in JavaScript, and use ML directly in the browser or in Node.js.

What is the Tensorflow.js model?

Tensorflow.js models are pre-trained models which means you need not to prepare/collect data to train the model. The models are hosted on NPM and unpkg so they can be used in any project out of the box.

In this blog, we are going to see one of the models in tensorflow.js which is the MediaPipe Facemesh model. The model is designed for front-facing cameras on mobile devices, where faces in view tend to occupy a relatively large fraction of the canvas.

Blazeface Model Demo

Image for post
Image for post
Demo

Demonstration

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-core"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-converter"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/blazeface"></script>

TensorFlow.js Core, flexible low-level API for neural networks and numerical computation.

TensorFlow.js Converter, tools to import a TensorFlow SavedModel to TensorFlow.js.

The blazeface package finds facial boundaries and landmarks within an image.

Step 2:- Include video HTML tag in the body content to read face through webcam.

<video width=640 height=480 autoplay muted id=”camera”></video>

Step 3:- Use canvas tag to draw graphics, on the fly, via scripting (usually JavaScript).

<canvas width=640 height=480 id=”augmented_canvas”></canvas>

Step 4:- Include a video tag to play video frames on canvas tag to move video frames as per face movement.

<video autoplay loop id=”movie” style=”visibility: hidden”>
<source src=”TensorFlowjs.mp4" type=”video/mp4"></source>
</video>

Step 5:- Load face model and estimate the face to finds facial boundaries and landmarks within an image.

//load camera stream
const frame = document.getElementById("camera");
//load movie stream
const movie = document.getElementById("movie");
movie.play();
//prepare canvas
const canvas = document.getElementById("augmented_canvas");
const draw = canvas.getContext("2d");
const result = await model.estimateFaces(frame, false);

Step 6:- Draw video frames on the detected face using the below code.

//copy camera stream to canvas
draw.drawImage(frame, 0, 0, 640, 480);
//check if face is detected
if(result.length > 0)
{
for (let i = 0; i < result.length; i++) {
const start = result[i].topLeft;
const end = result[i].bottomRight;
const size = [end[0] — start[0], end[1] — start[1]];
//Render a rectangle over each detected face.
draw.drawImage(movie, start[0], start[1], size[0], size[1]);
}

}

Conclusion

Everybody says if you want to become a data scientist you should familiarize in python or R programming language but now we can implement a machine learning application using javascript.

Image for post
Image for post
(Source: https://miro.medium.com/max/1400/1*F-1fq9TNjDnAYPAXnZP4Ww.png)

Happy Learning to all!! if you need any help or assistance please connect with me LinkedIn and Twitter.

I have a passion for understanding technology at a fundamental level and Sharing ideas and code. * Aspire to Inspire before I expire* https://balavenkatesh.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store