Book Image

Instant OpenCV for iOS

4 (1)
Book Image

Instant OpenCV for iOS

4 (1)

Overview of this book

Computer vision on mobile devices is becoming more and more popular. Personal gadgets are now powerful enough to process high-resolution images, stitch panoramas, and detect and track objects. OpenCV, with its decent performance and wide range of functionality, can be an extremely useful tool in the hands of iOS developers. Instant OpenCV for iOS is a practical guide that walks you through every important step for building a computer vision application for the iOS platform. It will help you to port your OpenCV code, profile and optimize it, and wrap it into a GUI application. Each recipe is accompanied by a sample project or an example that helps you focus on a particular aspect of the technology. Instant OpenCV for iOS starts by creating a simple iOS application and linking OpenCV before moving on to processing images and videos in real-time. It covers the major ways to retrieve images, process them, and view or export results. Special attention is also given to performance issues, as they greatly affect the user experience.Several computer vision projects will be considered throughout the book. These include a couple of photo filters that help you to print a postcard or add a retro effect to your images. Another one is a demonstration of the facial feature detection algorithm. In several time-critical cases, the processing speed is measured and optimized using ARM NEON and the Accelerate framework. OpenCV for iOS gives you all the information you need to build a high-performance computer vision application for iOS devices.
Table of Contents (7 chapters)

Working with images in Gallery (Intermediate)


In this recipe, we'll learn how to work with Gallery and will try to apply simple photo effects from the previous recipe to an image from Gallery. Also, it is the first recipe where we meet with the delegation pattern and actions (callbacks) for GUI elements.

If you run the corresponding project, you will get the following result:

Getting ready

Source code for this recipe can be found in the Recipe06_WorkingWithGallery folder in the code bundle that accompanies this book. You can use the iOS Simulator to work on this recipe, but you will also need an image with a face in your Gallery. You can open the Safari browser on the Simulator, copy lena.png from your PC using the drag-and-drop method, then save it using the long mouse click.

How to do it...

The first steps are the same as in the previous recipes. We should create an Xcode project, reference the OpenCV framework, add the UIImageView component, and copy files with the postcard printing code.

We are going to add the possibility to print a postcard with an image from Gallery (the image should have face in it) and save the resulting image back to Gallery. The following are the steps required to do it:

  1. Add UIToolbar and two UIBarButtonItem components to the GUI.

  2. Implement functions in our Controller that are relevant with needed interfaces.

  3. Create actions to respond to button-clicks.

  4. Finally, we will implement the printPostcard method that wraps the call to PostcardPrinter class. But before the postcard printing, we will preprocess the image with the preprocessFace method to add a vintage effect.

Let's implement the described steps:

  1. We will first update our GUI. In order to do it, we should add the UIToolbar component to the bottom part of the interface. As usual, we need to select the corresponding storyboard file, choose the Toolbar component from the Objects list and drag it to the View. This component already has one button. To rename it to Load, you should double-click on it and enter new text. After that, we have to add a Bar Button Item component from the Objects list for the second button and rename it to Save.

  2. The following is a declaration for our Controller interface from the ViewController.h file:

    @interface ViewController :
        UIViewController<UIImagePickerControllerDelegate,
                         UINavigationControllerDelegate,
                         UIPopoverControllerDelegate>
    {
            UIPopoverController* popoverController;
            UIImageView* imageView;
            UIImage* postcardImage;
            cv::CascadeClassifier faceDetector;
    }
    
    @property (nonatomic, strong) IBOutlet UIImageView* imageView;
    @property (nonatomic, strong) IBOutlet UIToolbar* toolbar;
    @property (nonatomic, strong) UIPopoverController* popoverController;
    @property (nonatomic, weak) IBOutlet UIBarButtonItem* loadButton;
    @property (nonatomic, weak) IBOutlet UIBarButtonItem* saveButton;
    
    -(IBAction)loadButtonPressed:(id)sender;
    -(IBAction)saveButtonPressed:(id)sender;
    
    - (UIImage*)printPostcard:(UIImage*)image;
    
    @end
  3. Then we should connect the IBOutlet properties of our Controller with corresponding components on GUI. Next, we'll consider implementation for some methods of the ViewController class needed to load images from Gallery:

    - (void)imagePickerController: (UIImagePickerController*)picker
        didFinishPickingMediaWithInfo:(NSDictionary *)info
    {
        if ([[UIDevice currentDevice] userInterfaceIdiom] ==
                                                UIUserInterfaceIdiomPad)
        {
            [popoverController dismissPopoverAnimated:YES];
        }
        else
        {
            [picker dismissViewControllerAnimated:YES
                                       completion:nil];
        }
        
        UIImage* temp =
            [info objectForKey:@"UIImagePickerControllerOriginalImage"];
        
        postcardImage = [self printPostcard:temp];
        imageView.image = postcardImage;
        
        [saveButton setEnabled:YES];
    }
    
    -(void)imagePickerControllerDidCancel:
        (UIImagePickerController *)picker
    {
        if ([[UIDevice currentDevice] userInterfaceIdiom] ==
                                                UIUserInterfaceIdiomPad)
        {
            [popoverController dismissPopoverAnimated:YES];
        }
        else
        {
            [picker dismissViewControllerAnimated:YES completion:nil];
        }
    }
  4. In order to add actions (callbacks) to our buttons, we have to implement two methods describing the response to clicks. You also should connect these functions with correspondent UI components:

    - (IBAction)loadButtonPressed:(id)sender
    {
        if (![UIImagePickerController isSourceTypeAvailable:
                        UIImagePickerControllerSourceTypePhotoLibrary])
            return;
        UIImagePickerController* picker =
            [[UIImagePickerController alloc] init];
        picker.delegate = self;
        picker.sourceType =
            UIImagePickerControllerSourceTypePhotoLibrary;
        
        if ([[UIDevice currentDevice] userInterfaceIdiom] ==
                                                UIUserInterfaceIdiomPad)
        {
            if ([self.popoverController isPopoverVisible])
            {
                [self.popoverController dismissPopoverAnimated:YES];
            }
            else
            {
                    self.popoverController =
                        [[UIPopoverController alloc]
                        initWithContentViewController:picker];
                    
                    popoverController.delegate = self;
                    
                    [self.popoverController
                     presentPopoverFromBarButtonItem:sender
                     permittedArrowDirections:UIPopoverArrowDirectionUp
                                     animated:YES];
            }
        }
        else
        {
            [self presentViewController:picker
                               animated:YES
                             completion:nil];
        }
    }
    
    - (IBAction)saveButtonPressed:(id)sender
    {
        if (postcardImage != nil)
        {
            UIImageWriteToSavedPhotosAlbum(postcardImage, self,
                                           nil, NULL);
            
            // Alert window
            UIAlertView *alert = [UIAlertView alloc];
            alert = [alert initWithTitle:@"Status"
                                 message:@"Saved to the Gallery!"
                                delegate:nil
                       cancelButtonTitle:@"Continue"
                       otherButtonTitles:nil];
            [alert show];
        }
    } 
  5. Finally, we should implement the printPostcard method that wraps the call to the PostcardPrinter class on the Objective-C side. You can do it yourself, similar to previous recipe. Please note that we also need to use the face detector to cut the face from the image and call the preprocessFace method to quantize intensity levels and add a vintage effect to the image.

The remaining functions (for example, viewDidLoad) were not changed much, so we'll not explain them in detail.

How it works...

For loading images from Gallery, we have to use the UIImagePickerController class. It provides user interfaces for choosing images and videos from your device. In order to use it, we should implement several protocols in our ViewController class, so it becomes a delegate that conforms to these protocols. This way, it follows the so-called delegation mechanism that is widely used in iOS. It allows you to avoid inheriting from base classes, and instead, the delegation requires implementing a protocol with some particular methods. In our recipe, we will use three delegates for taking images from Gallery: UIImagePickerControllerDelegate, UINavigationControllerDelegate, and UIPopoverControllerDelegate. To implement the first protocol, we should add the imagePickerControllerDidCancel method that will be called if the user presses the Cancel button before choosing an image in Gallery. In our recipe, we are just closing the window with the user's photo in this case. We should also implement the didFinishPickingMediaWithInfo method that describes the application behavior if the user selects an image. In our case, we call the printPostcard method for the selected image and store the result to the image global variable.

As you may have noticed, all these functions have a conditional statement that checks whether we use iPad or iPhone. It follows the UI guidelines for iOS. On iPhone and iPod devices, it is common to use a full screen window to show photos from Gallery, but on iPad, we have to use pop-up windows. So, to close the window, we should use two different implementations for corresponding classes of devices.

In the action of the Load button, we should create a UIImagePickerController object and set the delegate property of this object to our ViewController class. It allows us to inform our Controller about the changes and invoke the created implementation of the protocol. In the case of iPhone/iPod, we should just present the Controller with the presentViewController method. Implementation for tablets is a bit complicated; we should create a UIPopoverController object and initialize it with the previously created UIImagePickerController object. In order to do it, we will initialize the self.popoverController field that is already contained in our class, because we're using delegation from UIPopoverControllerDelegate.

It is our first recipe, where we were working with buttons. Here we deal with another important Cocoa design pattern called target-action. Actions are messages (the action) that are sent to the Controller (the target) on corresponding button-clicks. In order to process button-clicks, one should catch the corresponding events. For that purpose, you should use the IBAction keyword. IBAction is a special macro that resolves to void, but it denotes a method that can be linked with UI components.

There's more...

If you want to know more about the delegation and actions, we recommend you to read the Cocoa's Communicating with Objects guide at http://bit.ly/3848_CommunicatingWithObjects.

All the application logic is now in place, but we'll add some features to make our application more user-friendly.

Device orientation

Our postcards assume to be shown in the portrait orientation of a device. But, by default, GUI will be rotated if you rotate your device, and the image will be inadequately stretched. To avoid this effect, we can restrict the usage of undesirable orientations.

In order to do it, we can add the following function to our implementation of the ViewController class:

- (NSInteger)supportedInterfaceOrientations
{
    // Only portrait orientation
    return UIInterfaceOrientationMaskPortrait;
}

In this function, you should return a bit mask, which is a result of the bitwise OR operation for the desired orientations flags.

Disabling buttons

In our recipe, we are using the Save button to write the resulting image to Gallery. But we can't do it until we print our first postcard. In this situation, we can disable the button before the first image is chosen. To deactivate the button, we should use the setEnabled method:

[saveButton setEnabled:NO];