Book Image

Machine Learning with Core ML

Book Image

Machine Learning with Core ML

Overview of this book

Core ML is a popular framework by Apple, with APIs designed to support various machine learning tasks. It allows you to train your machine learning models and then integrate them into your iOS apps. Machine Learning with Core ML is a fun and practical guide that not only demystifies Core ML but also sheds light on machine learning. In this book, you’ll walk through realistic and interesting examples of machine learning in the context of mobile platforms (specifically iOS). You’ll learn to implement Core ML for visual-based applications using the principles of transfer learning and neural networks. Having got to grips with the basics, you’ll discover a series of seven examples, each providing a new use-case that uncovers how machine learning can be applied along with the related concepts. By the end of the book, you will have the skills required to put machine learning to work in their own applications, using the Core ML APIs
Table of Contents (16 chapters)
Title Page
Packt Upsell
Contributors
Preface
Index

Preface

We are living on the verge of a new era of computing, an era where computers are becoming more of a companion than a tool. The devices we carry in our pockets will soon better understand our world and us a lot better, and this will have a profound impact on how we interact with and use them.

But right now, a lot of these exciting advancements are stuck in the labs of researchers and not in the hands of designers and developers, making them usable and accessible to users. This is not because the details are locked away; on the contrary, in most cases they are freely available. 

This gap is somewhat due to our contentment with sticking to what we know, having the user do all the work, making them tap on the buttons. If nothing else, I hope this book makes you curious about what is out there and how it can be used to create new experiences, or improve existing ones. 

Within the pages of this book, you will find a series of examples to help you build an understanding of how deep neural networks work and how they can be applied.

This book focuses on a set of models for a better understanding of images and photos, specifically looking at how they can be adapted and applied on the iOS platform. This narrow focus of image-based models and the iOS platform is intentional; I find that the visual nature of images makes the concepts easier to, well, visualize, and the iPhone provides the perfect candidate and environment for experimentation.

So, as you go through this book, I encourage you to start thinking about new ways of how these models can be used and what new experiences you could create. With that being said, let's get started! 

Who this book is for

This book will appeal to three broad groups of people. The first are intermediate iOS developers who are interested in learning and applying machine learning (ML); some exposure to ML concepts may be beneficial but are not essential as this book covers the intuition behind the concepts and models used throughout it.

The second group are those who have experience in ML but not in iOS development and are looking for a resource to help them to get the grips with Core ML; for this group, it is recommended to complement this book with a book that covers the fundamentals of iOS development. 

The last group are experienced iOS developers and ML practitioners who are curious to see how various models have been applied in the context of the iOS platform. 

What this book covers

Chapter 1, Introduction to Machine Learning, provides a brief introduction to ML, including some explanation of the core concepts, the types of problems, algorithms, and general workflow of creating and using a ML models. The chapter concludes by exploring some examples where ML is being applied.

Chapter 2, Introduction to Apple Core ML, introduces Core ML, discussing what it is, what it is not, and the general workflow for using it.

Chapter 3, Recognizing Objects in the World, walks through building a Core ML application from start to finish. By the end of the chapter, we would have been through the whole process of obtaining a model, importing it into the project, and making use of it.

Chapter 4, Emotion Detection with CNNs, explores the possibilities of computers understanding us better, specifically our mood. We start by building our intuition of how ML can learn to infer your mood, and then put this to practice by building an application that does just that. We also use this as an opportunity to introduce the Vision framework and see how it complements Core ML. 

Chapter 5, Locating Objects in the World, goes beyond recognizing a single object to being able to recognize and locate multiple objects within a single image through object detection. After building our understanding of how it works, we move on to applying it to a visual search application that filters not only by object but also by composition of objects. In this chapter, we'll also get an opportunity to extend Core ML by implementing customer layers. 

Chapter 6, Creating Art with Style Transfer, uncovers the secrets behind the popular photo effects application, Prisma. We start by discussing how a model can be taught to differentiate between the style and content of an image, and then go on to build a version of  Prisma that applies a style from one image to another. We wrap up this chapter by looking at ways to optimize the model. 

Chapter 7, Assisted Drawing with CNNs, walks through building an application that can recognize a users sketch using the same concepts that have been introduced in previous chapters. Once what the user is trying to sketch has been recognized, we look at how we can find similar substitutes using the feature vectors from a CNN. 

Chapter 8, Assisted Drawing with RNNs, builds on the previous chapter and explores replacing the the convolution neural network (CNN) with a recurrent neural network (RNN) for sketch classification, thus introducing RNNs and showing how they can be applied to images. Along with a discussion on learning sequences, we will also delve into the details of how to download and compile Core ML models remotely. 

Chapter 9, Object Segmentation Using CNNs, walks through building an ActionShot photography application. And in doing so, we introduce another model and accompanying concepts, and get some hands-on experience of preparing and processing data.

Chapter 10, An Introduction to Create ML, is the last chapter. We introduce Create ML, a framework for creating and training Core ML models within Xcode using Swift. By the end of this chapter, you will know how to quickly create, train, and deploy a custom models. 

To get the most out of this book

To be able to follow through the examples in this book, you will need the following software:

  • macOS 10.13 or higher 
  • Xcode 9.2 or higher 
  • iOS 11.0 or higher (device and simulator) 

For the examples that are dependent on Core ML 2, you will need the following software:

  • macOS 10.14 
  • Xcode 10.0 beta 
  • iOS 12 (device and simulator)

It's recommended that you use https://notebooks.azure.com (or some other Jupyter notebook service provider) to follow the examples using the Core ML Tools Python package, but those wanting to run locally or train their model will need the following software:

  • Python 2.7 
  • Jupyter Notebooks 1.0
  • TensorFlow 1.0.0 or higher
  • NumPy 1.12.1 or higher
  • Core ML Tools 0.9 (and 2.0 for Core ML 2 examples)  

Download the example code files

You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

  1. Log in or register at www.packtpub.com.
  2. Select the SUPPORT tab.
  3. Click on Code Downloads & Errata.
  4. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

  • WinRAR/7-Zip for Windows
  • Zipeg/iZip/UnRarX for Mac
  • 7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Machine-Learning-with-Core-ML. In case there's an update to the code, it will be updated on the existing GitHub repository.

 We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/MachineLearningwithCoreML_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "At the top of the class, we have the VideoCaptureDelegate protocol defined."

A block of code is set as follows:

public protocol VideoCaptureDelegate: class {
    func onFrameCaptured(
      videoCapture: VideoCapture, 
      pixelBuffer:CVPixelBuffer?, 
      timestamp:CMTime)
}

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

@IBOutlet var previewView:CapturePreviewView!
@IBOutlet var classifiedLabel:UILabel!

let videoCapture : VideoCapture = VideoCapture()

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."

Note

Warnings or important notes appear like this.

Note

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packtpub.com.