With the basic understanding of how a CNN works from Chapter 1, Introduction to Deep Learning for Mobile, and how image processing is done at the most basic level, we are ready to proceed with using the pre-trained models from Firebase ML Kit to detect faces from the given images.

We will be using the Firebase ML Kit Face Detection API to detect the faces in an image. The key features of the Firebase Vision Face Detection API are as follows:

• Recognize and return the coordinates of facial features such as the eyes, ears, cheeks, nose, and mouth of every face detected.
• Get the contours of detected faces and facial features.
• Detect facial expressions, such as whether a person is smiling or has one eye closed.
• Get an identifier for each individual face detected in a video frame. This identifier is consistent across invocations and can be used to perform image manipulation on a particular face in a video stream.

Let's begin with the first step, adding the required dependencies. 

We start by adding the pub dependencies. A dependency is an external package that is required for a particular functionality to work. All of the required dependencies for the application are specified in the pubspec.yaml file. For every dependency, the name of the package should be mentioned. This is generally followed by a version number specifying which version of the package we want to use. Additionally, the source of the package, which tells pub how to locate the package, and any description that the source needs to find the package can also be included.

The dependencies that we will be using for this project are as follows:

• firebase_ml_vision: A Flutter plugin that adds support for the functionalities of Firebase ML Kit
• image_picker: A Flutter plugin that enables taking pictures with the camera and selecting images from Android or iOS image library

Here's what the dependencies section of the pubspec.yaml file will look like after including the dependencies:

dependencies:
  flutter:
    sdk: flutter
  firebase_ml_vision: ^0.9.2+1
  image_picker: ^0.6.1+4

Now we build the application. The application, named Face Detection, will consist of two screens. The first one will have a text title with two buttons, allowing the user to choose an image from the device's picture gallery or take a new image using the camera. After this, the user is directed to the second screen, which shows the image that was selected for face detection highlighting the detected faces. The following screenshot shows the flow of the application:

The widget tree of the application looks like this:

Let's now discuss the creation and implementation of each of the widgets in detail. 

Here we are creating the first screen. The user interface of the first screen will contain a text title, Pick Image, and two buttons, Camera and Gallery. This can be thought of as a column containing the text title and a row with two buttons, as shown in the following screenshot:

In the following sections, we will build each of these elements, called widgets, and then bring them together under a scaffold.

Then is building the row title. We start by creating a stateful widget, FaceDetectionHome, inside the face_detection_home.dart file. FaceDetectionHomeState will contain all the methods required to build the first screen of the application.

Let's define a method called buildRowTitle() to create the text header:

Widget buildRowTitle(BuildContext context, String title) {
    return Center(
        child: Padding(
            padding: EdgeInsets.symmetric(horizontal: 8.0, vertical: 16.0),
            child: Text(
                title,
                style: Theme.of(context).textTheme.headline,
            ), //Text
        ) //Padding
    ); //Center
}

The method is used to create a widget with a title using the value that is passed in the title string as an argument. The text is aligned to the center horizontally by using Center() and is provided a padding of 8.0 horizontally and 16.0 vertically using EdgeInsets.symmetric(horizontal: 8.0, vertical: 16.0). It contains a child, which is used to create the Text with the title. The typographical style of the text is modified to textTheme.headline to change the default size, weight, and spacing of the text.

Next is building the row with button widgets. After placing our text title, we will now create a row of two buttons that will enable the user to pick an image either from the gallery or take a new image from the camera. Let's do this in the following steps:

1. We start by defining createButton() to create buttons with all the required properties:

Widget createButton(String imgSource) {
    return Expanded(
        child: Padding(
            padding: EdgeInsets.symmetric(horizontal: 8.0),
            child: RaisedButton(
                color: Colors.blue,
                textColor: Colors.white,
                splashColor: Colors.blueGrey,
                onPressed: () {
                    onPickImageSelected(imgSource);
                },
                child: new Text(imgSource)
            ),
        )
    );
}

The method returns a widget, that is, RaisedButton, after providing a horizontal padding of 8.0. The color of the button is set to blue and the color of the button text is set to white. splashColor is set to blueGrey to indicate that the button is clicked by producing a rippling effect.

The code snippet inside onPressed is executed when the button is pressed. Here, we make a call to onPickImageSelected(), which is defined in a later section of the chapter. The text that is displayed inside the button is set to imgSource, which, here, can be the gallery or the camera. Additionally, the whole code snippet is wrapped inside Expanded() to make sure that the created button completely occupies all the available space.

2. Now we use the buildSelectImageRowWidget() method to build a row with two buttons to list the two image sources:

Widget buildSelectImageRowWidget(BuildContext context) {
    return Row(
        children: <Widget>[
            createButton('Camera'),
            createButton('Gallery')
        ],
    );
}

In the preceding code snippet, we call the previously defined createButton() method to add Camera and Gallery as image source buttons and add them to the children widget list for the row.

3. Now, let's define onPickImageSelected(). This method uses the image_picker library to direct the user either to the gallery or the camera to get an image:

void onPickImageSelected(String source) async {
    var imageSource;
    if (source == ‘Camera’) {
        imageSource = ImageSource.camera;
    } else {
        imageSource = ImageSource.gallery;
    }
    final scaffold = _scaffoldKey.currentState;
    try {
        final file = await ImagePicker.pickImage(source: imageSource);
        if (file == null) {
            throw Exception('File is not available');
        }
        Navigator.push(
            context,
            new MaterialPageRoute(
                builder: (context) => FaceDetectorDetail(file)),
            );
        } catch (e) {
        scaffold.showSnackBar(SnackBar(
        content: Text(e.toString()),
        ));
    }
}

First, imageSource is set to either camera or gallery using an if-else block. If the value passed is Camera, the source of the image file is set to ImageSource.camera; otherwise, it is set to ImageSource.gallery.

Once the source of the image is decided pickImage() is used to pick the correct imageSource. If the source was Camera, the user will be directed to the camera to take an image; otherwise, they will be directed to choose an image from the gallery.

To handle the exception if the image was not returned successfully by pickImage(), the call to the method is enclosed inside a try-catch block. If an exception occurs, the execution is directed to the catch block and a snackbar with an error message being shown on the screen by making a call to showSnackBar():

After the image has been chosen successfully and the file variable has the required uri, the user migrates to the next screen, FaceDetectorDetail, which is discussed in the section, Creating the second screen, and using Navigator.push() it passes the current context and the chosen file into the constructor. On the FaceDetectorDetail screen, we populate the image holder with the selected image and show details about the detected faces.

Now, we create the whole user interface, all of the created widgets are put together inside the build() method overridden inside the FaceDetectorHomeState class.

In the following code snippet, we create the final scaffold for the first screen of the application:

@override
Widget build(BuildContext context) {
    return Scaffold(
        key: _scaffoldKey,  
        appBar: AppBar(
            centerTitle: true,
            title: Text('Face Detection'),
        ),
        body: SingleChildScrollView(
            child: Column(
                children: <Widget>[
                    buildRowTitle(context, 'Pick Image'),
                    buildSelectImageRowWidget(context)
                ],
            )
        )
    );
}

The text of the toolbar is set to Face Detection by setting the title inside the appBar. Also, the text is aligned to the center by setting centerTitle to true. Next, the body of the scaffold is a column of widgets. The first is a text title and the next is a row of buttons.

Next, we create the second screen. After successfully obtaining the image selected by the user, we migrate to the second screen of the application, where we display the selected image. Also, we mark the faces that were detected in the image using Firebase ML Kit. We start by creating a stateful widget named FaceDetection inside a new Dart file, face_detection.dart.

First of all, the image that was selected needs to be passed to the second screen for analysis. We do this using the FaceDetection() constructor.

We declare a file variable and initialize it using a parameterized constructor as follows:

File file; 
FaceDetection(File file){
 this.file = file;
}

Now let's move on to the next step.

Now, we analyze the image to detect faces. We will create an instance of the FirebaseVision face detector to detect the faces using the following steps:

1. First, we create a global faces variable inside the FaceDetectionState class, as shown in the following code:

List<Face> faces;

2. Now we define a detectFaces() method, inside which we instantiate FaceDetector as follows:

void detectFaces() async{
    final FirebaseVisionImage visionImage = FirebaseVisionImage.fromFile(widget.file);
    final FaceDetector faceDetector = FirebaseVision.instance.faceDetector(FaceDetectorOptions( mode:     FaceDetectorMode.accurate, enableLandmarks: true, enableClassification: true));
    List<Face> detectedFaces = await faceDetector.processImage(visionImage);
    for (var i = 0; i < faces.length; i++) {
        final double smileProbablity = detectedFaces[i].smilingProbability;
        print("Smiling: $smileProb");
    }
    faces = detectedFaces;
 }

We first create a FirebaseVisionImage instance called visionImage of the image file that was selected using the FirebaseVisionImage.fromFile() method. Next, we create an instance of FaceDetector by using the FirebaseVision.instance.faceDetector() method and store it in a variable called faceDetector. Now we call processImage() using the FaceDetector instance, faceDetector, which was created earlier, and pass in the image file as a parameter. The method call returns a list of detected faces, which is stored in a list variable called detectedFaces. Note that processImage() returns a list of type Face. Face is an object whose attributes contain the characteristic features of a detected face. A Face object has the following attributes:

• getLandmark
• hashCode
• hasLeftEyeOpenProbability
• hasRightEyeOpenProbability
• headEulerEyeAngleY
• headEylerEyeAngleZ
• leftEyeOpenProbability
• rightEyeOpenProbability
• smilingProbability

Now we iterate through the list of faces using a for loop. We can get the value of smilingProbablity for the i^th face using detectedFaces[i].smilingProbability. We store it in a variable called smileProbablity and print its value to the console using print(). Finally, we set the value of the global faces list to detectedFaces.

To make sure that the faces are detected as soon as the user migrates to the second screen, we override initState() and call detectFaces() from inside it:

 @override
 void initState() {
     super.initState();
     detectFaces();
 }

Next, marking the detected faces. After detecting all the faces present in the image, we will paint rectangular boxes around them with the following steps:

1.  First we need to convert the image file into raw bytes. To do so, we define a loadImage method as follows:

void loadImage(File file) async {
    final data = await file.readAsBytes();
    await decodeImageFromList(data).then(
        (value) => setState(() {
        image = value;
        }),
    );
}

2. Now we define a CustomPainter class called FacePainter to paint rectangular boxes around all the detected faces. We start as follows:

class FacePainter extends CustomPainter {
    Image image;
    List<Face> faces;
    List<Rect> rects = [];
    FacePainter(ui.Image img, List<Face> faces) {
        this.image = img;
        this.faces = faces;
            for(var i = 0; i < faces.length; i++) {
                rects.add(faces[i].boundingBox);
            }
        }
    }
}

 @override
 void paint(Canvas canvas, Size size) {
     final Paint paint = Paint()
         ..style = PaintingStyle.stroke
         ..strokeWidth = 8.0
         ..color = Colors.red;
     canvas.drawImage(image, Offset.zero, Paint());
     for (var i = 0; i < faces.length; i++) {
         canvas.drawRect(rects[i], paint);
     }
 }

After successfully detecting faces and painting rectangles around them, we will now display the final image on the screen. We first build the final scaffold for our second screen. We will override the build() method inside FaceDetectionState to return the scaffold as follows:

 @override
 Widget build(BuildContext context) {
     return Scaffold(
         appBar: AppBar(
         title: Text("Face Detection"),
         ),
         body: (image == null) 
         ? Center(child: CircularProgressIndicator(),)
         : Center(
             child: FittedBox(
                 child: SizedBox(
                     width: image.width.toDouble(),
                     height: image.width.toDouble(),
                     child: CustomPaint(painter: FacePainter(image, faces))
                 ),
             ),
         )
     );
 }

At last, we create the final MaterialApp. We create the main.dart file, which provides the point of execution for the whole code. We create a stateless widget called FaceDetectorApp, which is used to return a MaterialApp specifying the title, theme, and home screen:

class FaceDetectorApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(
      debugShowCheckedModeBanner: false,
      title: 'Flutter Demo',
      theme: new ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: new FaceDetectorHome(),
    );
  }
}

Now we define the main() method to execute the whole application by passing in the instance of FaceDetectorApp() as follows:

void main() => runApp(new FaceDetectorApp());