Finding books in images using Python and OpenCV

Let’s start by taking a look at our example image we are going to count books in:
We see there are four books in the image, along with various “distractors” such as a coffee mug, a Starbucks cup, multiple coasters, and a piece of candy.
Our goal here is to find the four books in the image while ignoring the distractors.
How are we going to do that?
Read on to find out!

What libraries will we need?

In order to build our system to find and detect books in images, we’ll be utilizing two main libraries:

• NumPy to facilitate numerical operations.
• OpenCV to handle computer vision and image processing.

Make sure you have these libraries installed!

Finding books in images using Python and OpenCV.

Let’s go ahead and get started.
Open up your favorite code editor, create a new file named find_books.py, and let’s get started:

# import the necessary packages
import numpy as np
import cv2

# load the image, convert it to grayscale, and blur it
image = cv2.imread("example.jpg")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (3, 3), 0)
cv2.imshow("Gray", gray)
cv2.waitKey(0)

We’ll start by importing our required libraries. We’ll be using NumPy for numerical processing and cv2 for our OpenCV bindings.
Loading an image off disk is handled by the cv2.imread function. Here we are simply loading our image off disk, followed by converting it from the Red, Green, Blue (RGB) color space to grayscale.
We’ll also blur the image slightly to reduce high frequency noise and increase the accuracy of our code used to find books later in this post.
After executing our code, our output should look like this:

Here you can see that we have loaded the image off disk, converted it to grayscale, and blurred it slightly.
Now, let’s detect edges (i.e outlines) of the objects in the image:

# detect edges in the image
edged = cv2.Canny(gray, 10, 250)
cv2.imshow("Edged", edged)
cv2.waitKey(0)

Our edged image now looks like this:

We have clearly found the outlines of the objects in the images. However, you’ll notice that some of the outlines are not “clean” and complete. There are gaps in between the outlines that we need to close in order to successfully detect our books.
To solve this, we’ll apply a “closing” operation to close the gaps between the white pixels in the image:

# construct and apply a closing kernel to 'close' gaps between 'white'
# pixels
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7, 7))
closed = cv2.morphologyEx(edged, cv2.MORPH_CLOSE, kernel)
cv2.imshow("Closed", closed)
cv2.waitKey(0)

Sure enough, the gaps in the outlines have been closed:

The next step is to actually detect the outlines of the objects in the image. We’ll use the cv2.findContours function for that:

# find contours (i.e. the 'outlines') in the image and initialize the
# total number of books found
(cnts, _) = cv2.findContours(closed.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
total = 0

Let’s take a second and consider the geometry of a book.
A book is a rectangle. And a rectangle has four vertices. Therefore, if we examine a contour and find that it has four vertices, then we can assume it is a book and not one of the distractors in the image.
To check if a contour is a book or not, we need to loop over each of the contours individually:

# loop over the contours
for c in cnts:
# approximate the contour
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.02 * peri, True)

# if the approximated contour has four points, then assume that the
# contour is a book -- a book is a rectangle and thus has four vertices
if len(approx) == 4:
cv2.drawContours(image, [approx], -1, (0, 255, 0), 4)
total += 1

This code block is where all the magic happens. For each of the contours we compute the perimeter using cv2.arcLength and then approximate the contour using cv2.approxPolyDP.
The reason we approximate the contour is because the outline may not be a perfect rectangle. Due to noise when the photo was captured or shadows in the image, it is possible (and even very likely) that the book will not have exactly four vertices. By approximating the contour we can ensure we are able to side-step this problem.
Lastly, we make a check to see if the approximated contour does indeed have four vertices. If it does, then we draw the contour surrounding the book and then increment the total number of books counter.
We’ll wrap this example up by writing the total number of books found to the terminal and displaying the output image:

# display the output
print "I found {0} books in that image".format(total)
cv2.imshow("Output", image)
cv2.waitKey(0)

At this point our output image should look like this:

And our terminal does indeed show that we have successfully found the four books in the image will ignoring the other distractor objects:

To execute the script yourself, open up a terminal and execute the following commnd:

$ python find_books.py