Book Image

OpenCV 3 Blueprints

By : Joseph Howse, Puttemans, Sinha
Book Image

OpenCV 3 Blueprints

By: Joseph Howse, Puttemans, Sinha

Overview of this book

Computer vision is becoming accessible to a large audience of software developers who can leverage mature libraries such as OpenCV. However, as they move beyond their first experiments in computer vision, developers may struggle to ensure that their solutions are sufficiently well optimized, well trained, robust, and adaptive in real-world conditions. With sufficient knowledge of OpenCV, these developers will have enough confidence to go about creating projects in the field of computer vision. This book will help you tackle increasingly challenging computer vision problems that you may face in your careers. It makes use of OpenCV 3 to work around some interesting projects. Inside these pages, you will find practical and innovative approaches that are battle-tested in the authors’ industry experience and research. Each chapter covers the theory and practice of multiple complementary approaches so that you will be able to choose wisely in your future projects. You will also gain insights into the architecture and algorithms that underpin OpenCV’s functionality. We begin by taking a critical look at inputs in order to decide which kinds of light, cameras, lenses, and image formats are best suited to a given purpose. We proceed to consider the finer aspects of computational photography as we build an automated camera to assist nature photographers. You will gain a deep understanding of some of the most widely applicable and reliable techniques in object detection, feature selection, tracking, and even biometric recognition. We will also build Android projects in which we explore the complexities of camera motion: first in panoramic image stitching and then in video stabilization. By the end of the book, you will have a much richer understanding of imaging, motion, machine learning, and the architecture of computer vision libraries and applications!
Table of Contents (9 chapters)
8
Index

Shopping for glass

There is nothing quite like a well-crafted piece of glass that has survived many years and continued to sparkle because someone cared for it. On the mantle at home, my parents have a few keepsakes like this. One of them—a little, colorful glass flower—comes from a Parisian shopping mall where my brother and I ate many cheap but good meals during our first trip away from home and family. Other pieces go back earlier than I remember.

Some of the secondhand lenses that I use are 30 or 40 years old; their country of manufacture no longer exists; yet their glass and coatings are in perfect condition. I like to think that these lenses earned such good care by taking many fine pictures for previous owners and that they might still be taking fine pictures for somebody else 40 years from now.

Glass lasts. So do lens designs. For example, the Zeiss Planar T* lenses, one of the most respected names in the trade, are based on optical designs from the 1890s and coating processes from the 1930s. Lenses have gained electronic and motorized components to support autoexposure, autofocus, and image stabilization. However, the evolution of optics and coatings has been relatively slow. Mechanically, many old lenses are excellent.

Tip

Chromatic and spherical aberrations

An ideal lens causes all incoming rays of light to converge at one focal point. However, real lenses suffer from aberrations, such that the rays do not converge at precisely the same point. If different colors or wavelengths converge at different points, the lens suffers from chromatic aberrations, which are visible in the image as colorful haloes around high-contrast edges. If rays from the center and edge of the lens converge at different points, the lens suffers from spherical aberrations, which appear in the image as bright haloes around high-contrast edges in out-of-focus regions.

Starting in the 1970s, new manufacturing techniques have allowed for better correction of chromatic and spherical aberrations in high-end lenses. Apochromatic or "APO" lenses use highly refractive materials (often, rare earth elements) to correct chromatic aberrations. Aspherical or "ASPH" lenses use complex curves to correct spherical aberrations. These corrected lenses tend to be more expensive, but you should keep an eye out for them as they may sometimes appear at bargain prices.

Due to shallow depth of field, wide apertures produce the most visible aberrations. Even with non-APO and non-ASPH lenses, aberrations may be insignificant at most aperture settings, and in most scenes.

With a few bargain-hunting skills, we can find half a dozen good lenses from the 1960s to 1980s for the price of one good, new lens. Moreover, even recent lens models might sell at a 50 percent discount on the secondhand market. Before discussing specific examples of bargains, let's consider five steps we might follow when shopping for any secondhand lens:

  1. Understand the requirements. As we discussed earlier in the Capturing the subject in the moment section, many parameters of the lens and camera interact to determine whether we can take a clear and timely picture. At minimum, we should consider the appropriate focal length, f-number or T-number, and nearest focusing distance for a given application and given camera. We should also attempt to gauge the importance of high resolution and low distortion to the given application. Wavelengths of light matter, too. For example, if a lens is optimized for visible light (as most are), it is not necessarily efficient at transmitting NIR.
  2. Study the supply. If you live in a large city, perhaps local merchants have a good stock of used lenses at low prices. Otherwise, the lowest prices can typically be found on auction sites such as eBay. Search based on the requirements that we defined in step 1, such as "100 2.8 lens" if we are looking for a focal length of 100mm and an f-number or T-number of 2.8. Some of the lenses you find will not have the same type of mount as your camera. Check whether adapters are available. Adapting a lens is often an economical option, especially in the case of long lenses, which tend not to be mass-produced for cameras with a small sensor. Create a shortlist of the available lens models that seem to meet the requirements at an attractive price.
  3. Study the lens models. Online, have users published detailed specifications, sample images, test data, comparisons, and opinions? MFlenses (http://www.mflenses.com/) is an excellent source of information on old, manual focus lenses. It offers numerous reviews and an active forum. Image and video hosting sites such as Flickr (https://www.flickr.com/) are also good places to search for reviews and sample output of old and unusual lenses, including cine (video) lenses, nightvision scopes, and more! Find out about variations in each lens model over the years of its manufacture. For example, early versions of a given lens might be single-coated, while newer versions would be multicoated for better transmittance, contrast, and durability.
  4. Pick an item in good condition. The lens should have no fungus or haze. Preferably, it should have no scratches or cleaning marks (smudges in the coatings), though the effect on image quality might be small if the damage is to the front lens element (the element farthest from the camera).

    A little dust inside the lens should not affect image quality. The aperture and focusing mechanism should move smoothly and, preferably, there should be no oil on the aperture blades.

  5. Bid, make an offer, or buy at the seller's price. If you think the bidding or negotiation is reaching too high a price, save your money for another deal. Remember, despite what sellers might tell you, most bargain items are not rare and most bargain prices will occur again. Keep looking!

Some brands enjoy an enduring reputation for excellence. Between the 1860s and 1930s, German manufacturers such as Carl Zeiss, Leica, and Schneider Kreuznach solidified their fame as creators of premium optics. (Schneider Kreuznach is best known for cine lenses.) Other, venerable European brands include Alpa (of Switzerland) and Angéniuex (of France), which are both best known for cine lenses. By the 1950s, Nikon began to gain recognition as the first Japanese manufacturer to rival the quality of German lenses. Subsequently, Fuji and Canon became regarded as makers of high-end lenses for both cine and photo cameras.

Although Willy Loman (from Arthur Miller's play Death of a Salesman) might advise us to buy "a well-advertised machine", this is not necessarily the best bargain. Assuming we are buying lenses for their practical value and not their collectible value, we would be delighted if we found excellent quality in a mass-produced, no-name lens. Some lenses come reasonably close to this ideal.

East Germany and the Soviet Union mass-produced good lenses for photo cameras, cine cameras, projectors, microscopes, night vision scopes, and other devices. Optics were also important in major projects ranging from submarines to spacecraft! East German manufacturers included Carl Zeiss Jena, Meyer, and Pentacon. Soviet (later Russian, Ukrainian, and Belarusian) manufacturers included KMZ, BelOMO, KOMZ, Vologda, LOMO, Arsenal, and many others. Often, lens designs and manufacturing processes were copied and modified by multiple manufacturers in the Eastern Bloc, giving a recognizable character to lenses of this region and era.

Note

Many lenses from the Eastern Bloc were no-name goods insofar as they did not bear a manufacturer's name. Some models, for export, were simply labeled "MADE IN USSR" or "aus JENA" (from Jena, East Germany). However, most Soviet lenses bear a symbol and serial number that encode the location and date of manufacture. For a catalogue of these markings and a description of their historical significance, see Nathan Dayton's article, "An Attempt to Clear the FOG", at http://www.commiecameras.com/sov/.

Some of the lesser-known Japanese brands tend to be bargains, too. For example, Pentax makes good lenses but it has never commanded quite the same premium as its competitors do. The company's older photo lenses come in an M42 mount, and these are especially plentiful at low prices. Also, search for the company's C-mount cine lenses, which were formerly branded Cosmicar.

Let's see how we can couple some bargain lenses with the GS3-U3-23S6M-C to produce fine images. Remember that the GS3-U3-23S6M-C has a C mount and a sensor whose format is 1/1.2". For C-mount and CS-mount cameras, the name of the sensor format does not refer to any actual dimension of the sensor! Rather, for historical reasons, the measurement such as 1/1.2" refers to the diameter that a vacuum tube would have if video cameras still used vacuum tubes! The following table lists the conversions between common sensors formats and the actual dimensions of the sensor:

Format Name

Typical Lens Mounts

Typical Uses

Diagonal (mm)

Width (mm)

Height (mm)

Aspect Ratio

1/4"

CS

Machine vision

4.0

3.2

2.4

4:3

1/3"

CS

Machine vision

6.0

4.8

3.6

4:3

1/2.5"

C

Machine vision

6.4

5.1

3.8

4:3

1/2"

C

Machine vision

8.0

6.4

4.8

4:3

1/1.8"

C

Machine vision

9.0

7.2

5.4

4:3

2/3"

C

Machine vision

11.0

8.8

6.6

4:3

16mm

C

Cine

12.7

10.3

7.5

4:3

1/1.2"

C

Machine vision

13.3

10.7

8.0

4:3

Super 16

C

Cine

14.6

12.5

7.4

5:3

1"

C

Machine vision

16.0

12.8

9.6

4:3

Four Thirds

Four Thirds

Micro Four Thirds

Photography

Cine

21.6

17.3

13.0

4:3

APS-C

Various proprietary such as Nikon F

Photography

27.3

22.7

15.1

3:2

35mm

("Full Frame")

M42

Various proprietary such as Nikon F

Photography

Machine vision

43.3

36.0

24.0

3:2

Note

For more information about typical sensor sizes in machine vision cameras, see the following article from Vision-Doctor.co.uk: http://www.vision-doctor.co.uk/camera-technology-basics/sensor-and-pixel-sizes.html.

A lens casts a circular image, which needs to have a large enough diameter to cover the diagonal of the sensor. Otherwise, the captured image will suffer from vignetting, meaning that the corners will be blurry and dark. For example, vignetting is obvious in the following image of a painting, Snow Monkeys, by Janet Howse. The image was captured using a 1/1.2" sensor, but the lens was designed for a 1/2" sensor, and therefore the image is blurry and dark, except for a circular region in the center:

Shopping for glass

A lens designed for one format can cover any format that is approximately the same size or smaller, without vignetting; however, we might need an adapter to mount the lens. The system's diagonal field of view, in degrees, depends on the sensor's diagonal size and the lens's focal length, according to the following formula:

diagonalFOVDegrees =
  2 * atan(0.5 * sensorDiagonal / focalLength) * 180/pi

For example, if a lens has the same focal length as the sensor's diagonal, the diagonal FOV is 53.1 degrees. Such a lens is called a normal lens (with respect to the given sensor format) and a FOV of 53.1 degrees is considered neither wide nor narrow.

Consulting the table above, we see that the 1/1.2" format is similar to the 16mm and Super 16 formats. Thus, the GS3-U3-23S6M-C (and similar cameras) should be able to use most C-mount cine lenses without vignetting, without an adapter, and with approximately the same FOV as the lens designers intended.

Suppose we want a narrow FOV to capture a subject at a distance. We might need a long focal length that is not commonly available in C-mount. Comparing diagonal sizes, note that the 35mm format is larger than the 1/1.2" format by a factor of 3.25. A normal lens for 35mm format becomes a long lens with a 17.5 degree FOV when mounted for 1/1.2" format! An M42 to C mount adapter might cost $30 and lets us use a huge selection of lenses!

Suppose we want to take close-up pictures of a subject but none of our lenses can focus close enough. This problem has a low-tech solution—an extension tube, which increases the distance between the lens and camera. For C-mount, a set of several extension tubes, of various lengths, might cost $30. When the lens's total extension (from tubes plus its built-in focusing mechanism) is equal to the focal length, the subject is projected onto the sensor at 1:1 magnification. For example, a 13.3mm subject would fill the 13.3mm diagonal of a 1/1.2" sensor. The following formula holds true:

magnificationRatio = totalExtension / focalLength

However, for a high magnification ratio, the subject must be very close to the front lens element in order to come into focus. Sometimes, the use of extension tubes creates an impractical optical system that cannot even focus as far as its front lens element! Other times, parts of the subject and lens housing (such as a built-in lens shade) might bump into each other. Some experimentation might be required in order to determine the practical limits of extending the lens.

The following table lists some of the lenses and lens accessories that I have recently purchased for use with the GS3-U3-23S6M-C:

Name

Price

Origin

Mount and Intended Format

Focal Length (mm)

FOV for 1/1.2" Sensor (degrees)

Max f-number or T-number

Cosmicar TV Lens 12.5mm 1:1.8

$41

Japan

1980s?

C

Super 16

12.5

56.1

T/1.8

Vega-73

$35

LZOS factory

Lytkarino, USSR

1984

C

Super 16

20.0

36.9

T/2.0

Jupiter-11A

$50

KOMZ factory

Kazan, USSR

1975

M42

35mm

135.0

5.7

f/4.0

C-mount extension tubes: 10mm, 20mm, and 40mm

$30

Japan

New

C

-

-

-

Fotodiox M42 to C adapter

$30

USA

New

M42 to C

-

-

-

For comparison, consider the following table, which gives examples of new lenses that are specifically intended for the machine vision:

Name

Price

Origin

Mount and Intended Format

Focal Length (mm)

FOV for 1/1.2" Sensor (degrees)

Max f-number or T-number

Fujinon CF12.5HA-1

$268

Japan

New

C

1"

12.5

56.1

T/1.4

Fujinon CF25HA-1

$270

Japan

New

C

1"

25.0

29.9

T/1.4

Fujinon CF50HA-1

$325

Japan

New

C

1"

50.0

15.2

T/1.8

Fujinon CF75HA-1

$320

Japan

New

C

1"

75.0

10.2

T/1.4

Note

Prices in the preceding table are from B&H (http://www.bhphotovideo.com), a major photo and video supplier in the United States. It offers a good selection of machine vision lenses, often at lower prices than listed by industrial suppliers.

These new, machine vision lenses are probably excellent. I have not tested them. However, let's look at a few sample shots from our selection of old, used, photo and cine lenses that are cheaper by a factor of six or more. All the images are captured at a resolution of 1920x1200 using FlyCapture2, but they are resized for inclusion in this book.

First, let's try the Cosmicar 12.5mm lens. For a wide-angle lens such as this, a scene with many straight lines can help us judge the amount of distortion. The following sample shot shows a reading room with many bookshelves:

Shopping for glass

To better see the detail, such as the text on books' spines, look at the following crop of 480x480 pixels (25 percent of the original width) from the center of the image:

Shopping for glass

The following image is a close-up of blood vessels in my eye, captured with the Vega-73 lens and a 10mm extension tube:

Shopping for glass

Again, the detail can be better appreciated in a 480x480 crop from the center, as shown here:

Shopping for glass

Finally, let's capture a distant subject with the Jupiter-11A. Here is the Moon on a clear night:

Shopping for glass

Once more, let's examine a 480x480 crop from the image's center in order to see the level of detail captured by the lens:

Shopping for glass

By now, our discussion of bargain lenses has spanned many decades and the distance from here to the Moon! I encourage you to explore even further on your own. What are the most interesting and capable optical systems that you can assemble within a given budget?