Book Image

The Artificial Intelligence Infrastructure Workshop

By : Chinmay Arankalle, Gareth Dwyer, Bas Geerdink, Kunal Gera, Kevin Liao, Anand N.S.
Book Image

The Artificial Intelligence Infrastructure Workshop

By: Chinmay Arankalle, Gareth Dwyer, Bas Geerdink, Kunal Gera, Kevin Liao, Anand N.S.

Overview of this book

Social networking sites see an average of 350 million uploads daily - a quantity impossible for humans to scan and analyze. Only AI can do this job at the required speed, and to leverage an AI application at its full potential, you need an efficient and scalable data storage pipeline. The Artificial Intelligence Infrastructure Workshop will teach you how to build and manage one. The Artificial Intelligence Infrastructure Workshop begins taking you through some real-world applications of AI. You’ll explore the layers of a data lake and get to grips with security, scalability, and maintainability. With the help of hands-on exercises, you’ll learn how to define the requirements for AI applications in your organization. This AI book will show you how to select a database for your system and run common queries on databases such as MySQL, MongoDB, and Cassandra. You’ll also design your own AI trading system to get a feel of the pipeline-based architecture. As you learn to implement a deep Q-learning algorithm to play the CartPole game, you’ll gain hands-on experience with PyTorch. Finally, you’ll explore ways to run machine learning models in production as part of an AI application. By the end of the book, you’ll have learned how to build and deploy your own AI software at scale, using various tools, API frameworks, and serialization methods.
Table of Contents (14 chapters)
4. The Ethics of AI Data Storage

Using Vectorized Operations to Analyze Data Fast

The core building blocks in all programmers' toolboxes are looping and conditionals – usually materialized as a for loop or an if statement respectively. Almost any programming problem in its most fundamental form can be broken down into a series of conditional operations (only do something if a specific condition is met) and a series of iterative operations (carry on doing the same thing until a condition is met).

In machine learning, vectors, matrices, and tensors become the basic building blocks, taking over from arrays and linked lists. When we are manipulating and analyzing matrices, we often want to apply a single operation or function to the entire matrix.

Programmers coming from a traditional computer science background will often use a for loop or a while loop to do this kind of analysis or manipulation, but they are inefficient.

Instead, it is important to become comfortable with vectorized operations. Nearly all modern processors support efficiently modifying matrices and vectors in parallel by executing the same operation to each element simultaneously.

Similarly, many software packages are optimized for exactly this use case: applying the same operator to many rows of a matrix.

But if you are used to writing for loops, it can be difficult to get out of the habit. So, we will compare the for loop with the vectorized operation to help you understand the reason to avoid using a for loop. In the next exercise, we'll use our headlines dataset again and do some basic analysis. We'll do each piece of analysis twice: first using a for loop, and then again using a vectorized operation. You'll see the speed differences even on this relatively small dataset, but these differences will be even more important on the larger datasets that we previously discussed.

While some languages have great support for vectorized operations out of the box, Python relies mainly on third-party libraries to take advantage of these. We'll be using pandas in the upcoming exercise.

Exercise 1.02: Applying Vectorized Operations to Entire Matrices

In this exercise, we'll use the pandas library to load the same clickbait dataset and we'll carry out some descriptive analysis. We'll do each piece of analysis twice to see the efficiency gains of using vectorized operations compared to for loops.

Perform the following steps to complete the exercise:

  1. Create a new directory, Exercise01.02, in the Chapter01 directory to store the files for this exercise.
  2. Open your Terminal (macOS or Linux) or Command Prompt (Windows), navigate to the Chapter01 directory, and type jupyter notebook.
  3. In the Jupyter notebook, click the Exercise01.02 directory and create a new notebook file with a Python3 kernel.
  4. Import the pandas library and use it to read the dataset file into a DataFrame, as shown in the following code:
    import pandas as pd
    df = pd.read_csv("../Datasets/clickbait-headlines.tsv", \
                     sep="\t", names=["Headline", "Label"])

    You should get the following output:

    Figure 1.10: Sample of the dataset in a pandas DataFrame

    Figure 1.10: Sample of the dataset in a pandas DataFrame

    We import the pandas library and then use the read_csv() function to read the file into a DataFrame called df. We pass the sep argument to indicate that the file uses tab (\t) characters as separators and then pass in the column names as the names argument. The output is summarized to show only the first few entries and the last few, followed by a description of how many rows and columns there are in the entire DataFrame.

  5. Calculate the length of each headline and print out the first 10 lengths using a for loop, along with the total performance timing, as shown in the following code:
    lengths = []
    for i, row in df.iterrows():

    You should get the following output:

    [42, 60, 72, 49, 66, 51, 51, 58, 57, 76]
    CPU times: user 1.82 s, sys: 50.8 ms, total: 1.87 s
    Wall time: 1.95 s

    We declare an empty array to store the lengths, then loop through each row in our DataFrame using the iterrows() method. We append the length of the first item of each row (the headline) to our array, and finally, print out the first 10 results.

  6. Now re-calculate the length of each row, but this time using vectorized operations, as shown in the following code:
    lengths = df['Headline'].apply(len)

    You should get the following output:

    0    42
    1    60
    2    72
    3    49
    4    66
    5    51
    6    51
    7    58
    8    57
    9    76
    Name: Headline, dtype: int64
    CPU times: user 6.31 ms, sys: 1.7 ms, total: 8.01 ms
    Wall time: 7.76 ms

    We use the apply() function to apply len to every row in our DataFrame, without a for loop. Then we print the results to verify they are the same as when we used the for loop. From the output, we can see the results are the same, but this time it took only 16.3 milliseconds instead of over 1 second to carry out all of these calculations. Now, let's try a different calculation.

  7. This time, find the average length of all clickbait headlines and compare this average to the length of normal headlines, as shown in the following code:
    from statistics import mean
    normal_lengths = []
    clickbait_lengths = []
    for i, row in df.iterrows():
        if row[1] == 1:  # clickbait
    print("Mean normal length is {}"\
    print("Mean clickbait length is {}"\


    The # symbol in the code snippet above denotes a code comment. Comments are added into code to help explain specific bits of logic.

    You should get the following output:

    Mean normal length is 52.0322
    Mean clickbait length is 55.6876
    CPU times: user 1.91 s, sys: 40.7 ms, total: 1.95 s
    Wall time: 2.03 s

    We import the mean function from the statistics library. This time, we set up two empty arrays, one for the lengths of normal headlines and one for the lengths of clickbait headlines. We use the iterrows() function again to check every row and calculate the length, but this time store the result in one of our two arrays, based on whether the headline is clickbait or not. We then take the average of each array and print it out.

  8. Now recalculate this output using vectorized operations, as shown in the following code:
    print(df[df["Label"] == 0]['Headline'].apply(len).mean())
    print(df[df["Label"] == 1]['Headline'].apply(len).mean())

    You should get the following output:

    CPU times: user 10.5 ms, sys: 3.14 ms, total: 13.7 ms
    Wall time: 14 ms

    In each line, we look at only a subset of the DataFrame: first when the label is 0, and second when it is 1. We again apply the len function to each row that matches the condition and then take the average of the entire result. We confirm that the output is the same as before, but the overall time is in milliseconds in this case.

  9. As a final test, calculate how often the word "you" appears in each kind of headline, as shown in the following code:
    from statistics import mean
    normal_yous = 0
    clickbait_yous = 0
    for i, row in df.iterrows():
        num_yous = row[0].lower().count("you")
        if row[1] == 1:  # clickbait
            clickbait_yous += num_yous
            normal_yous += num_yous
    print("Total 'you's in normal headlines   {}".format(normal_yous))
    print("Total 'you's in clickbait headlines   {}".format(clickbait_yous))

    You should get the following output:

    Total 'you's in normal headlines 43
    Total 'you's in clickbait headlines 2527
    CPU times: user 1.48 s, sys: 8.84 ms, total: 1.49 s
    Wall time: 1.53 s

    We define two variables, normal_yous and clickbait_yous, to count the total occurrences of the word you in each class of headline. We loop through the entire dataset again using a for loop and the iterrows() function. For each row, we use the count() function to count how often the word you appear and then add this total to the relevant total. Finally, we print out both results, seeing that you appear very often in clickbait headlines, but hardly in non-clickbait headlines.

  10. Rerun the same analysis without using a for loop and compare the time, as shown in the following code:
    print(df[df["Label"] == 0]['Headline']\
            .apply(lambda x: x.lower().count("you")).sum())
    print(df[df["Label"] == 1]['Headline']\
            .apply(lambda x:   x.lower().count("you")).sum())

    You should get the following output:

    CPU times: user 20.8 ms, sys: 1.32 ms, total: 22.1 ms
    Wall time: 27.9 ms

    We break the dataset into two subsets and apply the same operation to each. This time, our function is a bit more complicated than the len function we used before, so we define an anonymous function inline using lambda. We lowercase each headline and count how often "you" appears and then sum the results. We notice that the performance time, in this case, is again in milliseconds.


    To access the source code for this specific section, please refer to

In this exercise, the main takeaway we can see is that vectorized operations can be many times faster than using for loops. We also learned some interesting things about clickbait characteristics though. For example, the word "you" appears very often in clickbait headlines (2,527 times), but hardly ever in normal headlines (43 times). Clickbait headlines are also, on average, slightly longer than non-clickbait headlines.

Let's implement the concepts learned so far in the next activity.

Activity 1.01: Creating a Text Classifier for Movie Reviews

In this activity, we will create another text classifier. Instead of training a machine learning model to discriminate between clickbait headlines and normal headlines, we will train a similar classifier to discriminate between positive and negative movie reviews.

The objectives of our activity are as follows:

  • Vectorize the text of IMDb movie reviews and label these as positive or negative.
  • Train an SVM classifier to predict whether a movie review is positive or negative.
  • Check how accurate our classifier is on a held-out test set.
  • Evaluate our classifier on out-of-context data.


    We will be using some randomizers in this activity. It is helpful to set the global random seeds to ensure that the results you see are the same as in the examples. Sklearn uses the NumPy random seed, and we will also use the shuffle function from the built-in random library. You can ensure you see the same results by adding the following code:

import random
import numpy as np

We'll use the aclImdb dataset of 100,000 movie reviews from Internet Movie Database (IMDb) – 50,000 each for training and testing. Each dataset has 25,000 positive reviews and 25,000 negative ones, so this is a larger dataset than our headlines one. The dataset can be found in our GitHub repository at the following location:

You need to download the aclImdb folder from the GitHub repository.

Dataset Citation: Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. (2011). Learning Word Vectors for Sentiment Analysis. The 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011).

In Exercise 1.01, Training a Machine Learning Model to Identify Clickbait Headlines, we had one file, with each line representing a different data item. Now we have a file for each data item, so keep in mind that we'll need to restructure some of our training code accordingly.


The code and the resulting output for this exercise have been loaded in a Jupyter notebook that can be found at

Perform the following steps to complete the activity:

  1. Import the os library and the random library, and define where our training and test data is stored using four variables: one for training_positive, one for training_negative, one for test_positive, and one for test_negative, each pointing at the respective dataset subdirectory.
  2. Define a read_dataset function that takes a path to a dataset and a label (either pos or neg) that reads the contents of each file in the given directory and adds these contents into a data structure that is a list of tuples. Each tuple contains both the text of the file and the label, pos or neg. An example is shown as follows. The actual data should be read from disk instead of being defined in code:
    contents_labels = [('this is the text from one of the files',   'pos'), ('this is another text', 'pos')]
  3. Use the read_dataset function to read each dataset into its variable. You should have four variables in total: train_pos, train_neg, test_pos, and test_neg, each one of which is a list of tuples, containing the relative text and labels.
  4. Combine the train_pos and train_neg datasets. Do the same for the test_pos and test_neg datasets.
  5. Use the random.shuffle function to shuffle the train and test datasets separately. This gives us datasets where the training data is mixed up, instead of feeding all the positive and then all the negative examples to the classifier in order.
  6. Split each of the train and test datasets back into data and labels respectively. You should have four variables again called train_data, y_train, test_data, and y_test where the y prefix indicates that the respective array contains labels.
  7. Import TfidfVectorizer from sklearn, initialize an instance of it, fit the vectorizer on the training data, and vectorize both the training and testing data into the X_train and X_test variables respectively. Time how long this takes and print out the shape of the training vectors at the end.
  8. Again, find the execution time, import LinearSVC from sklearn and initialize an instance of it. Fit the SVM on the training data and training labels, and then generate predictions on the test data (X_test).
  9. Import accuracy_score and classification_report from sklearn and calculate the results of your predictions. You should get the following output:
    Figure 1.11: Results – accuracy and the full report

    Figure 1.11: Results – accuracy and the full report

  10. See how your classifier performs on data on different topics. Create two restaurant reviews as follows:
    good_review = "The restaurant was really great! "\
                  "I ate wonderful food and had a very good time"
    bad_review = "The restaurant was awful. "\
                 "The staff were rude and "\
                 "the food was horrible. "\
                 "I hated it"
  11. Now vectorize each using the same vectorizer and generate predictions for whether each one is negative or positive. Did your classifier guess correctly?

Now that we've built two machine learning models and gained some hands-on experience with vectorized operations, it's time to recap.


The solution for this activity can be found via this link.