Book Image

Computer Vision for the Web

By : Foat Akhmadeev
Book Image

Computer Vision for the Web

By: Foat Akhmadeev

Overview of this book

This book will give you an insight into controlling your applications with gestures and head motion and readying them for the web. Packed with real-world tasks, it begins with a walkthrough of the basic concepts of Computer Vision that the JavaScript world offers us, and you’ll implement various powerful algorithms in your own online application. Then, we move on to a comprehensive analysis of JavaScript functions and their applications. Furthermore, the book will show you how to implement filters and image segmentation, and use tracking.js and jsfeat libraries to convert your browser into Photoshop. Subjects such as object and custom detection, feature extraction, and object matching are covered to help you find an object in a photo. You will see how a complex object such as a face can be recognized by a browser as you move toward the end of the book. Finally, you will focus on algorithms to create a human interface. By the end of this book, you will be familiarized with the application of complex Computer Vision algorithms to develop your own applications, without spending much time learning sophisticated theory.
Table of Contents (13 chapters)

Useful functions and where to use them


There are many functions that are needed in Computer Vision. Some of them are simple, such as sorting, while others are more complex. Here, we will discuss how to use them with the JSFeat library and see several Computer Vision applications.

Sorting using JSFeat

Sort algorithms are always helpful in any application. JSFeat provides an excellent way to sort a matrix. In addition to just sorting an array, it can even sort just part of the data. Let's see how we can do that:

  1. First, we need to define a compare function, which is as follows:

    var compareFunc = function (a, b) {
        return a < b;
    };
  2. Next, we do the sorting:

    var length = mat.data.length;
    jsfeat.math.qsort(mat.data, length / 3 * 2, length - 1, compareFunc);

The first parameter defines an array for sorting, the second and third are the starting index and the ending index, respectively. The final parameter defines the comparison function. You will see the following image:

As we can see, the lower portion part of the image was sorted, looks good!

You will probably need a median function, which returns the number that separates the higher part of the data from the lower part. To understand this better, we need to see some examples:

var arr1 = [2, 3, 1, 8, 5];
var arr2 = [4, 6, 2, 9, -1, 6];
var median1 = jsfeat.math.median(arr1, 0, arr1.length - 1);
var median2 = jsfeat.math.median(arr2, 0, arr2.length - 1);

For the first array, the result is 3. It is simple. For the sorted array, number 3 just separates 1, 2 from 5, 8. What we do see for the second array, is the result of 4. Actually, different median algorithms may return different results; for the presented algorithm, JSFeat picks one of the array elements to return the result. In contrast, many approaches will return 5 in that case, since 5 represents the mean of two middle values (4, 6). Taking that into account, be careful and see how the algorithm is implemented.

Linear algebra

Who wants to solve a system of linear equations? No one? Don't worry, it can be done very easily.

First, let's define a simple linear system. To start with, we define the linear system as Ax = B, where we know A and B matrices and need to find x:

var bufA = [9, 6, -3, 2, -2, 4, -2, 1, -2],
        bufB = [6, -4, 0];

var A = new jsfeat.matrix_t(3, 3, jsfeat.F32_t | jsfeat.C1_t, new jsfeat.data_t(bufA.length, bufA));
var B = new jsfeat.matrix_t(3, 1, jsfeat.F32_t | jsfeat.C1_t, new jsfeat.data_t(bufB.length, bufB));

jsfeat.linalg.lu_solve(A, B);

JSFeat places the result into the B matrix, so be careful if you want to use B somewhere else or you will loose your data. The result will look like this:

[2.000..., -4.000..., -4.000..]

Since the algorithm works with floats, we cannot get the exact values but after applying a round operation, everything will look fine:

[2, -4, -4]

In addition to this, you can use the svd_solve function. In that case, you will need to define an X matrix as well:

jsfeat.linalg.svd_solve(A, X, B);

A perspective example

Let us show you a more catchy illustration. Suppose you have an image that is distorted by perspective or you want to rectify an object plane, for example, a building wall. Here's an example:

Looks good, doesn't it? How do we do that? Let's look at the code:

var imgRectified = new jsfeat.matrix_t(mat.cols, mat.rows, jsfeat.U8_t | jsfeat.C1_t);
var transform = new jsfeat.matrix_t(3, 3, jsfeat.F32_t | jsfeat.C1_t);

jsfeat.math.perspective_4point_transform(transform,
        0, 0, 0, 0, // first pair x1_src, y1_src, x1_dst, y1_dst
        640, 0, 640, 0, // x2_src, y2_src, x2_dst, y2_dst and so on.
        640, 480, 640, 480,
        0, 480, 180, 480);
jsfeat.matmath.invert_3x3(transform, transform);
jsfeat.imgproc.warp_perspective(mat, imgRectified, transform, 255);

Primarily, as we did earlier, we define a result matrix object. Next, we assign a matrix for image perspective transformation. We calculate it based on four pairs of corresponding points. For example, the last, that is the fourth point of the original image, which is [0, 480], should be projected to the point of [180, 480] on the rectified image. Here, the first coordinate refers to X and the second to Y. Then, we invert the transform matrix to be able to apply it to the original image—mat variable. We pick the background color as white (255 for an unsigned byte). As a result, we get a nice image without any perspective distortion.