Face Recognition in Image using OpenCV-Python

Face recognition is an easy task for humans. we are already given the more tutorials about face detection, mouth detection, nose detection, eyes detection. one to three day old babies are able to distinguish between known faces. So how hard could it be for a computer? It turns out we know little about human recognition to date. Are inner features (eyes, nose, mouth) or outer features (head shape, hairline) used for a successful face recognition? How do we analyze an image and how does the brain encode it?. our brain has specialized nerve cells responding to specific local features of a scene, such as lines, edges, angles or movement. Since we don’t see the world as scattered pieces, our visual cortex must somehow combine the different sources of information into useful patterns. Automatic face recognition is all about extracting those meaningful features from an image, putting them into a useful representation and performing some kind of classification on them.
Face recognition based on the geometric features of a face is probably the most intuitive approach to face recognition. One of the first automated face recognition systems was described. marker points (position of eyes, ears, nose, …) were used to build a feature vector (distance between the points, angle between them, …). The recognition was performed by calculating the euclidean distance between feature vectors of a probe and reference image. Such a method is robust against changes in illumination by its nature, but has a huge drawback: the accurate registration of the marker points is complicated, even with state of the art algorithms. A 22-dimensional feature vector was used and experiments on large datasets have shown, that geometrical features alone my not carry enough information for face recognition. A facial image is a point from a high-dimensional image space and a lower-dimensional representation is found, where classification becomes easy. The lower-dimensional subspace is found with Principal Component Analysis, which identifies the axes with maximum variance. While this kind of transformation is optimal from a reconstruction standpoint, it doesn’t take any class labels into account. Imagine a situation where the variance is generated from external sources, let it be light. The axes with maximum variance do not necessarily contain any discriminative information at all, hence a classification becomes impossible.
Recently, I wanted to perform Face Recognition using OpenCV in Python but sadly, I could not find any good resource for the same. So, after a few hours of work, I wrote my own face recognition program using OpenCV and Python. The actual code is less than 40 lines of python code, thanks to the terse syntax of python and now, I am sharing with you what I did.
The whole process can be divided in three major steps –

1. The first step is to find a good database of faces with multiple images for each induvidual.
2. The next step is to detect faces in the database images and use them to train the face recognizer.
3. The last step is to test the face recognizer to recognize faces it was trained for.

So, let’s get started!

Database

We will use this database by using 9 images of the total 10 images of each individual in training our face recognizer and the remaining single image of each individual to test our face recognition algorithm.

Implementation

Now, we have an understanding of how our database looks like and it’s time to start programming the face recognition algorithm.

Import the required modules

The first step is to import the required modules –

1. cv2  – This is the OpenCV module and contains the functions for face detection and recognition.
2. os  – This module will be used to maneuver with image and directory names. First, we will use this module to extract the image names in the database directory and then from these names we will extract the individual number, which will be used as a label for the face in that image.
3. Image  – Since, the dataset images are in gif format and as of now, OpenCV does not support gif format, we will use Image module from PIL  to read the image in grayscale format.
4. numpy  – Our images will be stored in numpy arrays.
5.  sys – System initialization to system commands

import numpy as np
import cv2, os
import sys
from PIL import Image

Load the face detection Cascade

The first step is to detect the face in each image. Once, we get the region of interest containing the face in the image, we will use it for training the recognizer. For the purpose of face detection, we will use the Haar Cascade provided by OpenCV. The haar cascades that come with OpenCV are located in the /data/haarcascades> directory of your OpenCV installation. We will use haarcascade_frontalface_default.xml for detecting the face. So, we load the cascade using the cv2.CascadeClassifier function which takes the path to the cascade xml file. I have copied the xml file in the current working directory, so I have used the relative path. In case, you cannot locate the haar cascade file on your computer, I have included it in the zip file available for download at the bottom of the post.

cascadePath = "haarcascade_frontalface_default.xml face_cascade = cv2.CascadeClassifier(cascadePath)

Create the Face Recognizer Object

The next step is creating the face recognizer object. The face recognizer object has functions like FaceRecognizer.train to train the recognizer and FaceRecognizer.predict to recognize a face. OpenCV currently provides 3 face recognizers –

1. Eigenface Recognizer – createEigenFaceRecognizer()
2. Fisherface Recognizer  – createFisherFaceRecognizer()
3. Local Binary Patterns Histograms Face Recognizer – createLBPHFaceRecognizer()

We will use Local Binary Patterns Histograms Face Recognizer. So, let’s create the face recognizer –

recognizer = cv2.createBPHFaceRecognizer()

Create the function to prepare the training set

Now, we will define a function get_images_and_labels that takes the absolute path to the image database as input argument and returns tuple of 2 list, one containing the detected faces and the other containing the corresponding label for that face. For example, if the ith index in the list of faces represents the 5th individual in the database, then the corresponding ith location in the list of labels has value equal to 5.

def get_images_and_labels(path):
# Append all the absolute image paths in a list image_paths  
# We will not read the image with the .sad extension in the training set 
# Rather, we will use them to test our accuracy of the training - See 
image_paths = [os.path.join(path, f) for f in os.listdir(path) if not f.endswith('.sad')]
# images will contains face images
images = []
# labels will contains the label that is assigned to the image
labels = []
for image_path in image_paths:
    # Read the image and convert to grayscale
    image_pil = Image.open(image_path).convert('L   # Convert the image format into numpy array
    image = np.array(image_pil, 'uint8')
    # Get the label of the image
    nbr = int(os.path.split(image_path)[1].split(".")[0].replace("subject", "")) 
    # Detect the face in the image
    faces = faceCascade.detectMultiScale(image)
    # If face is detected, append the face to images and the label to labels
    for (x, y, w, h) in faces:
        images.append(image[y: y + h, x: x + w])
        labels.append(nbr)#sthash.wBCaaSGc.dpuf
    # return the images list and labels list
    return images, labels

In line 19, we are appending all the absolute path names of the database images in the list images_path. We, are not appending images with the .sad extension, as we will use them to test the accuracy of the recognizer. In line 21 and 23, we declare 2 list – images and labels. These are 2 list, that I had discussed in the previous paragraph that will be returned by the recognizer. In the list images, we append the region of interest containing the face and in the list labels, we append the corresponding label for that face. From line 24 – 38, we loop around each image to detect the face in it and update our 2 lists. So, in line 26 – 28 we load the current image in a 2D numpy array image. We cannot read the images directly using cv2.imread because as of now, OpenCV doesn’t support gif format images and unfortunately, our database images are in this format. So, we use the Image module from PIL to read the images in grayscale format and convert them into numpy arrays which are compatible with OpenCV. In** line 30, from the image name, we extract the individual number. This number will be the label for that face. **In line 32, we use CascadeClassifier.detectMultiScale to detect faces in the image. Although, in most cases, we need to tune the  CascadeClassifier.detectMultiScale function to correctly recognize faces in the image, but for sake of simplicity, I am leaving this part to the default values. You can refer to this Real Python post for more insights on this. The CascadeClassifier.detectMultiScale function returns a list of faces. For each face it returns a rectangle in the format (Top-Left x pixel value, Top-Left y pixel value, Width of rectangle, Height of rectangle.). In lines 34-38, we slice the ROI from the image and append it to the list images and the corresponding label in the list labels. Once, we are done with this loop, we return the 2 lists in the form of a tuple.
Detect the faces

for (x, y, w, h) in faces:
    cv2.rectangle(image, (x, y), (x+w, y+h), (0, 255, 0), 2)
    roi_gray = gray[y:y+h, x:x+w]
    roi_color = image[y:y+h, x:x+w]
    face_file_name = "img.jpg"
    cv2.imwrite(face_file_name, roi_color)

Load the faces from image:

    src = cv2.LoadImage("img.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src0 = cv2.LoadImage("img0.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src1 = cv2.LoadImage("img1.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src2 = cv2.LoadImage("img2.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src3 = cv2.LoadImage("img3.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src4 = cv2.LoadImage("img4.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src5 = cv2.LoadImage("img5.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src6 = cv2.LoadImage("img6.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src7 = cv2.LoadImage("img7.jpg", cv2.CV_LOAD_IMAGE_COLOR)
    src8 = cv2.LoadImage("img8.jpg", cv2.CV_LOAD_IMAGE_COLOR)

Load the faces from image what you are given the input.

Testing the face recognizer

    sc0= cv2.CompareHist(src, src0, cv2.CV_COMP_BHATTACHARYYA)
    sc1= cv2.CompareHist(src, src1, cv2.CV_COMP_BHATTACHARYYA)
    sc2= cv2.CompareHist(src, src2, cv2.CV_COMP_BHATTACHARYYA)
    sc3= cv2.CompareHist(src, src3, cv2.CV_COMP_BHATTACHARYYA)
    sc4= cv2.CompareHist(src, src4, cv2.CV_COMP_BHATTACHARYYA)
    sc5= cv2.CompareHist(src, src5, cv2.CV_COMP_BHATTACHARYYA)
    sc6= cv2.CompareHist(src, src6, cv2.CV_COMP_BHATTACHARYYA)
    sc7= cv2.CompareHist(src, src7, cv2.CV_COMP_BHATTACHARYYA)
    sc8= cv2.CompareHist(src, src8, cv2.CV_COMP_BHATTACHARYYA)

In above each instruction compare the image faces with database faces. cv2.CompareHist will return value. If the value is ‘0’ then the face is detected correctly. the person names will display on image.

    if sc0==0.0:
            cv2.putText(image, 'Raghava', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc1==0.0:
            cv2.putText(image, 'Prakash', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc2==0.0:
            cv2.putText(image, 'Praveen', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc3==0.0:
            cv2.putText(image, 'Ravi', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc4==0.0:
            cv2.putText(image, 'Prabhu', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc5==0.0:
            cv2.putText(image, 'Srikanth', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc6==0.0:
            cv2.putText(image, 'Sai', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc7==0.0:
            cv2.putText(image, 'Chaitu', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)
    if sc8==0.0:
            cv2.putText(image, 'Jagan', (x, y), cv2.FONT_ITALIC, 1, (200,255,155),2)

finally show the image with proper names

cv2.imshow("Face found" ,image)

Finally the output shown like bellow image: