Book Image

Test Driven Machine Learning

Book Image

Test Driven Machine Learning

Overview of this book

Table of Contents (16 chapters)
Test-Driven Machine Learning
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
2
Perceptively Testing a Perceptron
Index

Our first test


Let's start with an example of what a test looks like in Python. We will be using nosetests throughout this book. The main reason for using this is that while it is a bit of a pain to install a library, this library in particular will make everything that we do much simpler. The default unit test solution in Python requires a heavier set up. On top of this, by using nose, we can always mix in tests that use the built-in solution when we find that we need the extra features.

First, install it like this:

pip install nose

If you have never used pip before then it is time for you to know that it is a very simple way to install new Python libraries.

Now, as a hello world style example, let's pretend that we're building a class that will guess a number using the previous guesses to inform it. This is the simplest example to get us writing some code. We will use the TDD cycle that we discussed previously, and write our first test in painstaking detail. After we get through our first test and have something concrete to discuss, we will talk about the anatomy of the test that we wrote.

First, we must write a failing test. The simplest failing test that I can think of is the following:

def given_no_information_when_asked_to_guess_test():
  number_guesser = NumberGuesser()
  result = number_guesser.guess()
  assert result is None, "Then it should provide no result."

The context for the assert is in the test name. Reading the test name and then the assert name should do a pretty good job of describing what is being tested. Notice that in my test, I instantiate a NumberGuesser object. You're not missing any steps, this class doesn't exist yet. This seems roughly like how I'd want to use it. So, it's a great place to start, since it doesn't exist, wouldn't you expect this test to fail? Let's test this hypothesis.

To run the test, first make sure your test file is saved so that it ends in _tests.py. From the directory with the previous code, just run the following:

nosetests

When I do this, I get the following result:

There's a lot going on here, but the most informative part is near the end. The message is saying that NumberGuesser does not exist yet, which is exactly what I expected since we haven't actually written the code yet. Throughout the book, we'll reduce the detail of the stack traces that we show. For now, we'll keep things detailed to make sure that we're on the same page. At this point, we're in the "red" state of the TDD cycle:

  1. Next, create the following class in a file named NumberGuesser.py:

    class NumberGuesser:
      """Guesses numbers based on the history of your input"""
  2. Import the new class at the top of my test file with a simple import NumberGuesser statement.

  3. When I rerun nosetests, I get the following:

    TypeError: 'module' object is not callable

    Oh whoops! I guess that's not the right way to import the class. This is another very tiny step, but what is important is that we are making forward progress through constant communication with our tests. We are going through extreme detail because I can't stress this point enough. I will stop being as deliberate with this in the following chapter, bear with me for the time being.

  4. Change the import statement to the following:

    from NumberGuesser import NumberGuesser
  5. Rerun nosetests and you will see the following:

    AttributeError: NumberGuesser instance has no attribute 'guess'
  6. The error message has changed, and is leading to the next thing that needs to be changed. From here, we just implement what we think we need for the test to pass:

    class NumberGuesser:
      """Guesses numbers based on the history of your input"""
      def guess(self):
        return None
  7. On rerunning the nosetests, we'll get the following result:

That's it! Our first successful test! Some of these steps seem so tiny so as to not be worthwhile. Indeed, overtime, you may decide that you prefer to work on a different level of detail. For the sake of argument, we'll be keeping our steps pretty small, if only to illustrate just how much TDD keeps us on track and guides us on what to do next. We all know how to write the code in very large, uncontrolled steps. Learning to code surgically requires intentional practice, and is worth doing explicitly. Let's take a step back and look at what this first round of testing took.

The anatomy of a test

Starting from a higher level, notice how I had a dialog with Python. I just wrote the test and Python complained that the class that I was testing didn't exist. Next, I created the class, but then Python complained that I didn't import it correctly. So, then I imported it correctly, and Python complained that my "guess" method didn't exist. In response, I implemented the way that my test expected, and Python stopped complaining.

This is the spirit of TDD. There is a conversation between yourself and your system. You can work in steps as little or as large as you're comfortable with. What I did previously could've been entirely skipped over, though the Python class could have been written and imported correctly the first time. The longer you go without "talking" to the system, the more likely you are to stray from the path of getting things working as simply as possible.

Let's zoom in a little deeper and dissect this simple test to see what makes it tick. Here is the same test, but I've commented it, and broken it into sections that you will see recurring in every test that you write:

def given_no_information_when_asked_to_guess_test():
  # given
  number_guesser = NumberGuesser()
  # when
  guessed_number = number_guesser.guess()
  # then
  assert guessed_number is None, 'there should be no guess.'

Given

This section sets up the context for the test. In the previous test, you saw that I didn't provide any prior information to the object. In many of our machine learning tests, this will be the most complex portion of our test. We will be importing certain sets of data, sometimes making a few specific issues in the data and testing our software to handle the details that we would expect. When you think about this section of your tests, try to frame it as "Given this scenario…". In our test, we might say "Given no prior information for NumberGuesser…".

When

This should be one of the simplest aspects of our test. Once you've set up the context, there should be a simple action that triggers the behavior that you want to test. When you think about this section of your tests, try to frame it as "When this happens…". In our test we might say "When NumberGuesser guesses a number…".

Then

This section of our test will check on the state of our variables and any returned results, if applicable. Again, this section should also be fairly straightforward, as there should be only a single action that causes a change to your object under the test. The reason for this is that if it takes two actions to form a test, then it is very likely that we will just want to combine the two into a single action that we can describe in terms that are meaningful in our domain. A key example may be loading the training data from a file and training a classifier. If we find ourselves doing this a lot, then why not just create a method that loads data from a file for us?

As we progress through this book, you will find examples where we'll have the helper functions help us determine whether our results have changed in certain ways. Typically, we should view these helper functions as code smells. Remember that our tests are the first applications of our software. Anything that we have to build in addition to our code, to understand the results, is something that we should probably (there are exceptions to every rule) just include in the code we are testing.

"Given, When, Then" is not a strong requirement of TDD, because our previous definition of TDD only consisted of two things (all that the code requires is a failing test first and to eliminate duplication). We will still follow this convention in this book because:

  • Following some conventions throughout the book will make it much more readable.

  • It is the culmination of the thoughts of many people who were beginning to see patterns in how they were using TDD. This is a technique that has changed how I approach testing, so I use it here.

It's a small thing to be passionate about and if it doesn't speak to you, just translate this back into "Arrange, Act, Assert" in your head. At the very least, consider it as well as why these specific, very deliberate words are used.