After installation, preparing a new OpenCV code project is quite a straightforward process that requires including the header files and instructing the compiler to find the files and libraries used in the project.

OpenCV is composed of several modules, grouping related functionalities. Each module has an associated header file (for example, core.hpp) located in the directory with the same name as that of the module (that is, OPENCV_BUILD\install\include\opencv2\<module>). The supplied modules with the current version of OpenCV are as follows:

All OpenCV classes and functions are in the cv namespace. Consequently, we will have the following two options in our source code:

The DataType class defines the primitive datatypes for OpenCV. The primitive datatypes can be bool, unsigned char, signed char, unsigned short, signed short, int, float, double, or a tuple of values of one of these primitive types. Any primitive type can be defined by an identifier in the following form:

CV_<bit depth>{U|S|F}C(<number of channels>)

In the preceding code, U, S, and F stand for unsigned, signed, and float respectively. For the single channel arrays, the following enumeration is applied describing the datatypes:

enum {CV_8U=0, CV_8S=1, CV_16U=2, CV_16S=3, CV_32S=4, CV_32F=5, CV_64F=6};

The following diagram shows a graphical representation of a single channel (4 x 4) array with 8 bits of unsigned integers (CV_8U). In this case, each element should have a value from zero to 255, which may be represented by a grayscale image.

Single channel array of 8 bit unsigned integers for a greyscale image

We can define all of the preceding datatypes for multichannel arrays (up to 512 channels). The following diagram illustrates a graphical representation of three channels 4 x 4 array of 8 bits of unsigned integers (CV_8UC3). In this example, the array consists of tuples of three elements corresponding to an RGB image.

A three-channel array of 8 bit unsigned integers for an RGB image

The OpenCV Mat class is used for dense n-dimensional single or multichannel arrays. It can store real or complex-valued vectors and matrices, colored or grayscale images, histograms, point clouds, and so on. There are many different ways to create a Mat object, the most popular being the constructor where the size and type of the array are specified as follows:

Mat(nrows, ncols, type[, fillValue])

The initial value for the array elements might be set by the Scalar class as a typical four-element vector (for the RGB and transparency components of the image stored in the array). Next, we show some usage examples of Mat as follows:

Mat img_A(640, 480, CV_8U, Scalar(255)); // white image
// 640 x 480 single-channel array with 8 bits of unsigned integers
// (up to 255 values, valid for a grayscale image, for example,
// 255=white)
…
Mat img_B(Size(800, 600), CV_8UC3, Scalar(0,255,0)); // Green image
// 800 x 600 three channel array with 8 bits of unsigned integers
// (up to 24 bits color depth, valid for a RGB color image)

The Mat class is the main data structure that stores and manipulates images. OpenCV has implemented mechanisms to allocate and release memory automatically for these data structures. However, the programmer should still take special care when data structures share the same buffer memory.

Many functions in OpenCV process dense single or multichannel arrays usually using the Mat class. However, in some cases, a different datatype may be convenient, such as std::vector<>, Matx<>, Vec<>, or Scalar. For this purpose, OpenCV provides the proxy classes, InputArray and OutputArray, which allow any of the previous types to be used as parameters for functions.

To prepare the examples for this book, we used the Qt Creator IDE (included in the Qt 5.2 bundle) and OpenCV 2.4.9 compiled with MinGW g++ 4.8 and Qt functionality. Qt Creator is a free multiplatform IDE with very helpful features for C++ programming. However, the user can choose the tool chain to build the executables that best fit its needs.

Our first Qt Creator project with OpenCV will be quite a simple flip-image tool, named flipImage. This tool reads a color image file and transforms it into a grayscale image, flipped and saved into an output file.

For this application, we choose to create a new code project by navigating to File | New File or File | Project…, and then navigate to Non-Qt Project | Plain C++ Project. Then, we have to choose a project name and location. The next step is to pick a kit (that is, compiler) for the project (in our case, Desktop Qt 5.2.1 MinGW 32 bit) and location for the binaries generated. Usually, two possible build configurations (profiles) are used: debug and release. These profiles set the appropriate flags to build and run the binaries.

When a Qt Creator project is created, two special files (with the .pro and .pro.user extension) are generated to configure the build and run processes. The build process is determined by the kit chosen during the creation of the project. With the Desktop Qt 5.2.1 MinGW 32 bit kit, this process relies on the qmake and mingw32-make tools. With the .pro files as input, qmake generates the makefiles for Make (that is, mingw32-make) that drive the build process for each profile (that is, release and debug).

The qmake project file

For our flipImage sample project, the flipImage.pro file looks like the following code:

TARGET: flipImage
TEMPLATE = app
CONFIG += console
CONFIG -= app_bundle
CONFIG -= qt

SOURCES += \
    flipImage.cpp
INCLUDEPATH += C:\\opencv-buildQt\\install\\include
LIBS += -LC:\\opencv-buildQt\\install\\x64\mingw\\lib \
    -lopencv_core249.dll \
    -lopencv_highgui249.dll

The preceding file illustrates the options that qmake needs to generate the appropriate makefiles to build the binaries for our project. Each line starts with a tag indicating an option (TARGET, CONFIG, SOURCES, INCLUDEPATH, and LIBS) followed with a mark to add (+=) or remove (-=) the value of the option. In this sample project, we deal with the non-Qt console application. The executable file is flipImage.exe (TARGET) and the source file is flipImage.cpp (SOURCES). Since this project is an OpenCV application, the two last tags point out to the location of the header files (INCLUDEPATH) and the OpenCV libraries (LIBS) used by this particular project (for example, core and highgui). Note that a backslash at the end of the line denotes continuation in the next line. In Windows, path backslashes should be duplicated, as shown in the preceding example.

The following code shows the source code for the flipImage project:

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <iostream>

using namespace std;
using namespace cv;

int main(int argc, char *argv[])
{
    int flip_code=0;
    Mat out_image; // Output image

    if (argc != 4) {//Check args. number
        cout << "Usage: <cmd> <flip_code> <file_in> <file_out>\n";
        return -1;
    }
    Mat in_image = imread(argv[2], CV_LOAD_IMAGE_GRAYSCALE);
    if (in_image.empty()) { // Check if read
        cout << "Error! Input image cannot be read...\n";
        return -1;
    }
    sscanf(argv[1], "%d", &flip_code); // Read flip code
    flip(in_image, out_image, flip_code);
    imwrite(argv[3], out_image); // Write image to file
    namedWindow("Flipped…"); // Creates a window
    imshow(win, out_image); // Shows output image on window
    cout << "Press any key to exit...\n";
    waitKey(); // Wait infinitely for key press
    return 0;
}

After building the project, we can run the flipImage application from the following command line:

CV_SAMPLES/flipImage_build/debug>flipImage.exe -1 lena.jpg lena_f.jpg

The following screenshot shows the window with the output image after flipping on both the axes (horizontal and vertical):

Input image (left) and output image after the flipImage tool has been applied (right)

The source code starts with the inclusion of the header files (core.hpp and highgui.hpp) associated with the modules used by the application. Note that it is also possible to include only the opencv.hpp header since it will in turn include all the header files of OpenCV.

The flipImage example gets the flip code and two file names (for the input and output images) as the command-line arguments. These arguments are obtained from the argv[] variable. The following example illustrates several essential tasks in an OpenCV application:

1. Read an image from the file (imread) to a Mat class and check whether the target variable is not empty (Mat::empty).
2. Call a procedure (for example, flip) with the proxy classes, InputArray (in_image) and OutputArray (out_image).
3. Write an image to a file (imwrite).
4. Create an output window (namedWindow) and show (imshow) an image on it.
5. Wait for a key (waitKey).

The code explanation is given as follows:

• Mat imread(const string& filename, int flags=1): This function loads an image from the specified file and returns it. It also returns an empty matrix if the image cannot be read. It supports the most usual image formats of the files, detected by their content rather than by their extension. The flags parameter indicates the color of the image loaded in the memory, which may differ from the original color of the image stored in the file. In the example code, this function is used as follows:

  Mat in_image = imread(argv[2], CV_LOAD_IMAGE_GRAYSCALE);

  Here, the filename is obtained from the command-line arguments (the second argument after the command name). The CV_LOAD_IMAGE_GRAYSCALE flag indicates that the image should be loaded in the memory as an 8 bit grayscale image. For a description of the available tags, it is recommended to read the OpenCV online documentation (available at http://docs.opencv.org/).

• bool imwrite(const string& filename, InputArray img, const vector<int>& params=vector<int>()): This function writes an image to a given file where some optional format parameters are specified after the second argument. The format of the output file is determined by the file extension. In our example code, this function is used without the format parameters as follows:

  imwrite(argv[3], out_image);

• void namedWindow(const string& winname, int flags=WINDOW_AUTOSIZE): This function creates a window without displaying it. The first argument is a string used as a name for the window and its identifier. The second argument is a flag or flag combination, which controls some window properties (for example, enable resize). Next, we show how this function is used in the example using a constant string as a name for the created window, as follows:

  namedWindow("Flipped …"); // Creates a window

  Compiling OpenCV with Qt adds some new functionality to the highgui module (more on that later). Then, the window created with Qt and the namedWindow function uses the default flags: CV_WINDOW_AUTOSIZE, CV_WINDOW_KEEPRATIO, or CV_GUI_EXPANDED.

• void imshow(const string& winname, InputArray mat): This function displays an array (image) in a window with the properties set previously with the specified flags when the window was created. In the example, this function is used as follows:

  imshow(win, out_image); // Shows output image on window

• int waitKey(int delay=0): This function waits for a key press or the milliseconds specified by delay (if delay is greater than zero). If delay is less than or equal to zero, it waits infinitely. It returns the key code if pressed or -1 if a key is not pressed after the delay. This function has to be used after creating and activating a window. In the example code, it is used as follows:

  waitKey(); // Wait infinitely for key press

A video deals with moving images rather than still images, that is, display of a frame sequence at a proper rate (FPS or frames per second). The following showVideo example illustrates how to read and play a video file with OpenCV:

//… (omitted for simplicity)
int main(int argc, char *argv[])
{
    Mat frame; // Container for each frame

    VideoCapture vid(argv[1]); // Open original video file
    if (!vid.isOpened()) // Check whether the file was opened
        return -1;
    int fps = (int)vid.get(CV_CAP_PROP_FPS);
    namedWindow(argv[1]); // Creates a window
    while (1) {
        if (!vid.read(frame)) // Check end of the video file
            break;
        imshow(argv[1], frame); // Show current frame on window
        if (waitKey(1000/fps) >= 0)
            break;
    }
    return 0;
}

The code explanation is given as follows:

In the preceding example, the waitKey function is used with a computed number of milliseconds (1000/fps) trying to play the video file at the same rate it was originally recorded. Playing a video at a faster/slower rate (more/less fps) than that will produce a faster/slower playback.

Usually, the computer vision problems we face are related with processing live video input from one or several cameras. In this section, we will describe the recLiveVid example, which grabs a video stream from a webcam (connected to our computer), displays the stream in a window, and records it in a file (recorded.avi). By default, in the following example, the video capture is taken from the camera with cam_id=0. However, it is possible to handle a second camera (cam_id=1) and grab the video from it, setting an argument at the command line:

//… (omitted for brevity)
int main(int argc, char *argv[])
{
    Mat frame;
    const char win_name[]="Live Video...";
    const char file_out[]="recorded.avi";
    int cam_id=0; // Webcam connected to the USB port
    double fps=20;

    if (argc == 2)
        sscanf(argv[1], "%d", &cam_id);

    VideoCapture inVid(cam_id); // Open camera with cam_id
    if (!inVid.isOpened())
        return -1;

    int width = (int)inVid.get(CV_CAP_PROP_FRAME_WIDTH);
    int height = (int)inVid.get(CV_CAP_PROP_FRAME_HEIGHT);
    VideoWriter recVid(file_out, CV_FOURCC('F','F','D','S'), fps, Size(width, height));
    if (!recVid.isOpened()) 
        return -1;

    namedWindow(win_name);
    while (1) {
        inVid >> frame; // Read frame from camera
        recVid << frame; // Write frame to video file
        imshow(win_name, frame); // Show frame
        if (waitKey(1000/fps) >= 0)
            break;
    }
    inVid.release(); // Close camera
    return 0;
}

The code explanation is given as follows:

This chapter started with an explanation of how to build and install the OpenCV library with Qt (using CMake, the GNU g++ compiler, and GNU Make). Then, it is given a quick look to the modules organization of the library with an easy explanation of its basic API concepts. The chapter follows up with a more detailed revision of the fundamental data structures to store arrays and manipulate images. Also, three examples of code, such as flipImage, showVideo , and recLiveVid are explained to illustrate the basic use of the OpenCV library. The next chapter will introduce you to the two mainstream options available to provide graphical user interface capabilities for OpenCV programs.