Cartoon Effect is a technique that converts an image into a cartoon by applying few filters. In computer vision performing such operations is quite common and OpenCV a module in computer vision provides features for applying cartoon effects on images.
For converting an image into a cartoon we need to perform smoothing of the image. The first step is to apply some blurriness to the image such that noises will be removed. To remove these edges there are different in-built functions provided by the OpenCV.
Before jumping directly into blurring images, it is better we get to know the real concept behind how the blurring works. Blurring works on normalizing the value of the pixel by calculating the average value of the neighboring pixel.
Averaging
This is done by convolving an image with a normalized box filter. It simply takes the average of all the pixels under the kernel area and replaces the central element. Thus each pixel value in an image is replaced by the weighted average of the neighborhood intensity values. The kernel area is defined by a filter mask and the most commonly used filter is the box filter.
This is done by the function cv2.blur() or cv2.boxFilter().
There are different techniques available and the most effective among them are Gaussian blur, median blur. Now let’s see how the averaging concept is used in Gaussian blur and Median Blur.
Gaussian Blur
Gaussian Blur uses the concept of weighted averaging, which means the concept of the kernel area is used but the weight of the pixels is calculated based on the distance from the center of the kernel.
The farther the pixel from the center the less effect it has on the weighted average. This weighted average is applied to modify the pixel at the center. We should specify the width and height of the kernel which should be positive and odd.
We also should specify the standard deviation in the X and Y directions, sigma X and sigma Y respectively. To perform gaussian blur OpenCV provides an in-built method known as cv2.GaussianBlur()
cv2.GaussianBlur(src, ksize, sigmaX, dst, sigmaY, borderType)Median Blur
In the median blur, the center pixel is replaced by the calculated median of the neighboring pixels. The difference between Gaussian blur and Median blur is that changing the pixel value.
Since in Gaussian blur the new pixel value might or might not be in the neighboring pixels, because it is a weighted average.
But in the median blur, the central pixel value is replaced by one of the neighboring pixels itself by calculating the median value among them. Thus it is more effective when it comes to reducing the noise from the images.
The kernel size has to be odd and +ve.
median = cv2.medianBlur(img,5)Bilateral Filter in OpenCV
A bilateral filter is a non-linear, edge-preserving, and noise-reducing smoothing filter for images. It replaces the intensity of each pixel with a weighted average of intensity values from nearby pixels. This weight can be based on a Gaussian distribution.
The main property of Bilateral Filtering is that it does not do averaging across edges. That is why it is also called an edge-preserving filter.
cv2.bilateralFilter(), is highly effective at noise removal while preserving edges. Bilateral filtering also takes a Gaussian filter in space, but additionally considers one more Gaussian filter which is a function of pixel difference.
Gaussian Function of Space ensures that for the blurring effect only the spatial neighbors are considered, on the other hand, Gaussian Function of Intensity makes sure the pixels with similar intensities are considered for normalization.
Mathematically Bilateral Filter is defined by:-
- I(filtered) is the filtered image
- I is the original input image to be filtered
- x are the coordinates of the current pixel to be filtered
- is the window centered in
, so
is another pixel
is the range kernel for smoothing differences in intensities (this function can be a Gaussian function
is the spatial (or domain) kernel for smoothing differences in coordinates (this function can be a Gaussian function)
Thus Bilateral Filter preserves the edges and removes extra noise from the image by performing the following:-
- Replacing each pixel by a weighted average of neighbouring pixels.
- Each neighbouring pixel is weighted by a spatial component that acts as a filter for differentiating distant pixels and nearby pixels.
- The combination of both the components ensures that only nearby similar pixels contribute to the weighted average.
We can apply the Bilateral filter using the command cv2.bilateralFilter()
cv2.bilateralFilter(src, d, sigmaColor, sigmaSpace)- src It is the image whose is to be blurred
- d Diameter of each pixel neighbourhood that is used during filtering. If it is non-positive, it is computed from sigmaSpace.
- sigmaColor Filter sigma in the colour space. A larger value of the parameter means that farther colours within the pixel neighbourhood will be mixed together, resulting in larger areas of semi-equal colour.
- sigmaSpace Filter sigma in the coordinate space. A larger value of the parameter means that farther pixels will influence each other as long as their colours are close enough. When d>0, it specifies the neighbourhood size regardless of sigmaSpace. Otherwise, d is proportional to sigmaSpace.
- Border type border mode is used to extrapolate pixels outside of the image
Adaptive Thresholding
Simple thresholding is global and applies the thresholding effect on all the pixels of the image, irrespective of the fast illumination effects. Due to variation in illumination, there is a chance of false prediction and might result in false output.
To overcome this kind of false predictions and illumination problems on the images, there is a concept known as Adaptive thresholding.
Here the algorithm determines the threshold for a pixel-based on a small region around it. So we get different thresholds for different regions of the same image which gives better results for images with varying illumination.
cv2.AdaptiveThreshold(src, dst, maxValue, adaptive_method, thresholdType, blocksize, C)Parameters
- src − an 8-bit image representing the source. It has to be a grey-scale image.
- maxValue − This is the value we will be assigned to pixels that have value more than the threshold value
- Adaptive method −. Normalizing the central pixel based on the adaptive method. This will be either of the following two values
- ADAPTIVE_THRESH_MEAN_C − threshold value is the mean of the neighborhood area.
- ADAPTIVE_THRESH_GAUSSIAN_C − threshold value is the weighted sum of neighborhood values where weights are a Gaussian window.
- Threshold type − Threshold type for applying the thresholding on pixels, we can choose
THRESH_BINARYORTHRESH_BINARY_INV. - blockSize − A variable of the integer type representing the size of the pixel neighborhood is used to calculate the threshold value also called as kernel size.
- C − A variable of double type representing the constant used in both methods (subtracted from the mean or weighted mean).
Now implementing all the concepts cv2.medianBlur(), cv2.adaptiveThreshold(), cv2.bilateralFilter() to apply cartoon effects to the image. The process for applying cartoon effects is pretty easy but learning the concepts involved in the transformation is the main criteria for this article.
import cv2
import numpy as np
# reading source file
img = cv2.imread('people.jpg')
# converting the image into gray-scale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img2 = cv2.medianBlur(img, 1)
#applying adaptive threshold to use it as a mask
edges = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 9, 9)
color = cv2.bilateralFilter(img, 9, 200, 200)
#cartoonize
cartoon = cv2.bitwise_and(color, color, mask = edges)
cv2.imshow(cartoon)
cv2.waitKey(0)
cv2.destroyAllWindows()
