Detecting Faces Using a Haar Cascade Classifier

Have you ever wondered how Facebook automatically tags images with your friends' faces? Or how high-end cameras automatically find and focus on a certain person's face? Applications like these depend heavily on the machine learning task known as face detection - which is the task of automatically finding faces in images containing people.

At its root face detection is a classification problem - that is a problem of distinguishing between distinct classes of things. With face detection these distinct classes are 1) images of human faces and 2) everything else.

Haar Cascade classifier can also be trained to detect other things like for example eyes.

OpenCV is an open source library for Python and C++ with support for image manipulations and machine learning algorithms.

We use OpenCV's implementation of Haar feature-based cascade classifiers to detect human faces in images, OpenCV provides many pre-trained face detectors, stored as XML files on GitHub so it’s possible to pick the preferred one based on the case study and hardware constraints.

Note that by default OpenCV assumes the ordering of the image's color channels are Blue, then Green, then Red (BGR) and not the classic RGB to which we are all used to.

This face detector uses information about patterns of intensity in an image to reliably detect faces under varying light conditions. So, to use this Haar Cascade detector, it’s better to first convert the image from color to grayscale.

Note that even though the image is a black and white image, the data could have color information so it will still need to be converted to grayscale in order to perform the most accurate face detection.

Make a facial keypoint detector

Facial keypoints (also called facial landmarks) mark important areas of the face - the eyes, corners of the mouth, the nose, etc.

Facial keypoints can be used in a variety of machine learning applications from face and emotion recognition to commercial applications like Apple Face ID or the image filters popularized by Snapchat.

how can we make a facial keypoint detector? Well, at a high level, notice that facial keypoint detection is a regression problem. For example, a single face could correspond to a set of 15 facial keypoints (a set of 15 correspondings (x,y)(x,y) coordinates, i.e., an output point).

Because our input data are images, we can employ a convolutional neural network to recognize patterns in our images and learn how to identify these keypoint given sets of labelled data.

Conclusion

Facial keypoints are the starting point for the creation of an algorithm for facial encoding, able to create a representation of the unique features of a face, once encoded it can be used with other machine learning techniques to identify matches with sample images, the most simple solution is by using Euclidean distances between encodings establishing an arbitrary tolerance to consider a match valid.

If you find this article interesting please join our mailing list to not miss any update and schedule a free no obligation meeting with us to discuss your AI and Computer Vision needs.

Cheers, Andrea