OpenCV 3.x with Python By Example: Make the most of OpenCV and Python to build applications for object recognition and augmented reality , Second Edition

Convolution is a fundamental operation in image processing. We basically apply a mathematical operator to each pixel, and change its value in some way. To apply this mathematical operator, we use another matrix called a kernel. The kernel is usually much smaller in size than the input image. For each pixel in the image, we take the kernel and place it on top so that the center of the kernel coincides with the pixel under consideration. We then multiply each value in the kernel matrix with the corresponding values in the image, and then sum it up. This is the new value that will be applied to this position in the output image.

Here, the kernel is called the image filter and the process of applying this kernel to the given image is called image filtering. The output obtained after applying the kernel to the image is called the filtered image. Depending on the values in the kernel, it performs different functions such as blurring, detecting edges, and so on. The following figure...

Blurring refers to averaging the pixel values within a neighborhood. This is also called a low pass filter. A low pass filter is a filter that allows low frequencies, and blocks higher frequencies. Now, the next question that comes to our mind is: what does frequency mean in an image? Well, in this context, frequency refers to the rate of change of pixel values. So we can say that the sharp edges would be high-frequency content because the pixel values change rapidly in that region. Going by that logic, plain areas would be low-frequency content. Going by this definition, a low pass filter would try to smooth the edges.

A simple way to build a low pass filter is by uniformly averaging the values in the neighborhood of a pixel. We can choose the size of the kernel depending on how much we want to smooth the image, and it will correspondingly have different effects. If you choose a bigger size, then you will be averaging over a larger area. This tends to increase the smoothing effect...

When we apply the motion blurring effect, it will look like you captured the picture while moving in a particular direction. For example, you can make an image look like it was captured from a moving car.

The input and output images will look like the following ones:

Following is the code to achieve this motion blurring effect:

import cv2 
import numpy as np 
 
img = cv2.imread('images/input.jpg') 
cv2.imshow('Original', img) 
 
size = 15 
 
# generating the kernel 
kernel_motion_blur = np.zeros((size, size)) 
kernel_motion_blur[int((size-1)/2), :] = np.ones(size) 
kernel_motion_blur = kernel_motion_blur / size 
 
# applying the kernel to the input image 
output = cv2.filter2D(img, -1, kernel_motion_blur) 
 
cv2.imshow('Motion Blur', output) 
cv2.waitKey(0)

Applying the sharpening filter will sharpen the edges in the image. This filter is very useful when we want to enhance the edges of an image that's not crisp enough. Here are some images to give you an idea of what the image sharpening process looks like:

As you can see in the preceding figure, the level of sharpening depends on the type of kernel we use. We have a lot of freedom to customize the kernel here, and each kernel will give you a different kind of sharpening. To just sharpen an image, as we are doing in the top-right image in the preceding picture, we would use a kernel like this:

If we want to do excessive sharpening, as in the bottom-left image, we would use the following kernel:

But the problem with these two kernels is that the output image looks artificially enhanced. If we want our images to look more natural, we would use an edge enhancement filter. The underlying concept remains the same, but we use an approximate Gaussian kernel to build this filter. It will help...

An embossing filter will take an image and convert it to an embossed image. We basically take each pixel, and replace it with a shadow or a highlight. Let's say we are dealing with a relatively plain region in the image. Here, we need to replace it with a plain gray color because there's not much information there. If there is a lot of contrast in a particular region, we will replace it with a white pixel (highlight), or a dark pixel (shadow), depending on the direction in which we are embossing.

This is what it will look like:

Let's take a look at the code and see how to do this:

import cv2 
import numpy as np 
 
img_emboss_input = cv2.imread('images/input.jpg') 
 
# generating the kernels 
kernel_emboss_1 = np.array([[0,-1,-1], 
                            [1,0,-1], 
                            [1,1,0]]) 
kernel_emboss_2 = np.array([[-1,-1,0], 
                            [-1,0,1], 
                            [0,1,1]]) 
kernel_emboss_3 = np.array([[1,0,0], 
                   ...

The process of edge detection involves detecting sharp edges in the image, and producing a binary image as the output. Typically, we draw white lines on a black background to indicate those edges. We can think of edge detection as a high pass filtering operation. A high pass filter allows high-frequency content to pass through and blocks the low-frequency content. As we discussed earlier, edges are high-frequency content. In edge detection, we want to retain these edges and discard everything else. Hence, we should build a kernel that is the equivalent of a high pass filter.

Let's start with a simple edge detection filter known as the Sobel filter. Since edges can occur in both horizontal and vertical directions, the Sobel filter is composed of the following two kernels:

The kernel on the left detects horizontal edges and the kernel on the right detects vertical edges. OpenCV provides a function to directly apply the Sobel filter to a given image. Here is the code to use Sobel...

Erosion and dilation are morphological image processing operations. Morphological image processing basically deals with modifying geometric structures in the image. These operations are primarily defined for binary images, but we can also use them on grayscale images. Erosion basically strips out the outermost layer of pixels in a structure, whereas dilation adds an extra layer of pixels to a structure.

Let's see what these operations look like:

Following is the code to achieve this:

import cv2 
import numpy as np 
 
img = cv2.imread('images/input.jpg', 0) 
 
kernel = np.ones((5,5), np.uint8) 
 
img_erosion = cv2.erode(img, kernel, iterations=1) 
img_dilation = cv2.dilate(img, kernel, iterations=1) 
 
cv2.imshow('Input', img) 
cv2.imshow('Erosion', img_erosion) 
cv2.imshow('Dilation', img_dilation) 
 
cv2.waitKey(0) 

Using all the information we have, let's see if we can create a nice vignette filter. The output will look something like the following:

Here is the code to achieve this effect:

import cv2 
import numpy as np 
 
img = cv2.imread('images/input.jpg') 
rows, cols = img.shape[:2] 
 
# generating vignette mask using Gaussian kernels 
kernel_x = cv2.getGaussianKernel(cols,200) 
kernel_y = cv2.getGaussianKernel(rows,200) 
kernel = kernel_y * kernel_x.T 
mask = 255 * kernel / np.linalg.norm(kernel) 
output = np.copy(img) 
 
# applying the mask to each channel in the input image 
for i in range(3): 
    output[:,:,i] = output[:,:,i] * mask 
 
cv2.imshow('Original', img) 
cv2.imshow('Vignette', output) 
cv2.waitKey(0) 

Whenever we capture images in low-light conditions, the images turn out to be dark. This typically happens when you capture images in the evening, or in a dimly lit room. You must have seen this happen many times! The reason this happens is because the pixel values tend to concentrate near zero when we capture the images under such conditions. When this happens, a lot of details in the image are not clearly visible to the human eye. The human eye likes contrast, and so we need to adjust the contrast to make the image look nice and pleasant. A lot of cameras and photo applications implicitly do this already. We use a process called histogram equalization to achieve this.

To give an example, this is what it looks like before and after contrast enhancement:

As we can see here, the input image on the left is really dark. To rectify this, we need to adjust the pixel values so that they are spread across the entire spectrum of values, that is, between 0-255.

Following...

In this chapter, we learned how to use image filters to apply cool visual effects to images. We discussed the fundamental image processing operators, and how we can use them to build various things. We learned how to detect edges using various methods. We understood the importance of 2D convolution and how we can use it in different scenarios. We discussed how to smooth, motion-blur, sharpen, emboss, erode, and dilate an image. We learned how to create a vignette filter, and how we can change the region of focus as well. We discussed contrast enhancement and how we can use histogram equalization to achieve it.

In the next chapter, we will discuss how to cartoonize a given image.