Book Image

Distributed Computing with Go

By : V.N. Nikhil Anurag
Book Image

Distributed Computing with Go

By: V.N. Nikhil Anurag

Overview of this book

Distributed Computing with Go gives developers with a good idea how basic Go development works the tools to fulfill the true potential of Golang development in a world of concurrent web and cloud applications. Nikhil starts out by setting up a professional Go development environment. Then you’ll learn the basic concepts and practices of Golang concurrent and parallel development. You’ll find out in the new few chapters how to balance resources and data with REST and standard web approaches while keeping concurrency in mind. Most Go applications these days will run in a data center or on the cloud, which is a condition upon which the next chapter depends. There, you’ll expand your skills considerably by writing a distributed document indexing system during the next two chapters. This system has to balance a large corpus of documents with considerable analytical demands. Another use case is the way in which a web application written in Go can be consciously redesigned to take distributed features into account. The chapter is rather interesting for Go developers who have to migrate existing Go applications to computationally and memory-intensive environments. The final chapter relates to the rather onerous task of testing parallel and distributed applications, something that is not usually taught in standard computer science curricula.
Table of Contents (11 chapters)

Testing in Go

Testing is an important part of programming, whether it is in Go or in any other language. Go has a straightforward approach to writing tests, and in this section, we will look at some important tools to help with testing.

There are certain rules and conventions we need to follow to test our code. They can be listed as follows:

  • Source files and associated test files are placed in the same package/folder
  • The name of the test file for any given source file is <source-file-name>_test.go
  • Test functions need to have the "Test" prefix, and the next character in the function name should be capitalized

In the remainder of this section, we will look at three files and their associated tests:

  • variadic.go and variadic_test.go
  • addInt.go and addInt_test.go
  • nil_test.go (there isn't any source file for these tests)

Along the way, we will introduce any further concepts we might use.

variadic.go

In order to understand the first set of tests, we need to understand what a variadic function is and how Go handles it. Let's start with the definition:

Variadic function is a function that can accept any number of arguments during function call.

Given that Go is a statically typed language, the only limitation imposed by the type system on a variadic function is that the indefinite number of arguments passed to it should be of the same data type. However, this does not limit us from passing other variable types. The arguments are received by the function as a slice of elements if arguments are passed, else nil, when none are passed.

Let's look at the code to get a better idea:

// variadic.go 
 
package main 
 
func simpleVariadicToSlice(numbers ...int) []int { 
   return numbers 
} 
 
func mixedVariadicToSlice(name string, numbers ...int) (string, []int) { 
   return name, numbers 
} 
 
// Does not work. 
// func badVariadic(name ...string, numbers ...int) {} 

We use the ... prefix before the data type to define a functions as a variadic function. Note that we can have only one variadic parameter per function and it has to be the last parameter. We can see this error if we uncomment the line for badVariadic and try to test the code.

variadic_test.go

We would like to test the two valid functions, simpleVariadicToSlice and mixedVariadicToSlice, for various rules defined in the previous section. However, for the sake of brevity, we will test these:

  • simpleVariadicToSlice: This is for no arguments, three arguments, and also to look at how to pass a slice to a variadic function
  • mixedVariadicToSlice: This is to accept a simple argument and a variadic argument

Let's now look at the code to test these two functions:

// variadic_test.go 
package main 
 
import "testing" 
 
func TestSimpleVariadicToSlice(t *testing.T) { 
    // Test for no arguments 
    if val := simpleVariadicToSlice(); val != nil { 
        t.Error("value should be nil", nil) 
    } else { 
        t.Log("simpleVariadicToSlice() -> nil") 
    } 
 
    // Test for random set of values 
    vals := simpleVariadicToSlice(1, 2, 3) 
    expected := []int{1, 2, 3} 
    isErr := false 
    for i := 0; i < 3; i++ { 
        if vals[i] != expected[i] { 
            isErr = true 
            break 
        } 
    } 
    if isErr { 
        t.Error("value should be []int{1, 2, 3}", vals) 
    } else { 
        t.Log("simpleVariadicToSlice(1, 2, 3) -> []int{1, 2, 3}") 
    } 
 
    // Test for a slice 
    vals = simpleVariadicToSlice(expected...) 
    isErr = false 
    for i := 0; i < 3; i++ { 
        if vals[i] != expected[i] { 
            isErr = true 
            break 
        } 
    } 
    if isErr { 
        t.Error("value should be []int{1, 2, 3}", vals) 
    } else { 
        t.Log("simpleVariadicToSlice([]int{1, 2, 3}...) -> []int{1, 2, 3}") 
    } 
} 
 
func TestMixedVariadicToSlice(t *testing.T) { 
    // Test for simple argument & no variadic arguments 
    name, numbers := mixedVariadicToSlice("Bob") 
    if name == "Bob" && numbers == nil { 
        t.Log("Recieved as expected: Bob, <nil slice>") 
    } else { 
        t.Errorf("Received unexpected values: %s, %s", name, numbers) 
    } 
} 

Running tests in variadic_test.go

Let's run these tests and see the output. We'll use the -v flag while running the tests to see the output of each individual test:

$ go test -v ./{variadic_test.go,variadic.go}                                                                                                              
=== RUN   TestSimpleVariadicToSlice        
--- PASS: TestSimpleVariadicToSlice (0.00s)                                           
        variadic_test.go:10: simpleVariadicToSlice() -> nil                           
        variadic_test.go:26: simpleVariadicToSlice(1, 2, 3) -> []int{1, 2, 3}         
        variadic_test.go:41: simpleVariadicToSlice([]int{1, 2, 3}...) -> []int{1, 2, 3}                                                                                      
=== RUN   TestMixedVariadicToSlice         
--- PASS: TestMixedVariadicToSlice (0.00s) 
        variadic_test.go:49: Received as expected: Bob, <nil slice>                   
PASS                                       
ok      command-line-arguments  0.001s    

addInt.go

The tests in variadic_test.go elaborated on the rules for the variadic function. However, you might have noticed that TestSimpleVariadicToSlice ran three tests in its function body, but go test treats it as a single test. Go provides a good way to run multiple tests within a single function, and we shall look them in addInt_test.go.

For this example, we will use a very simple function as shown in this code:

// addInt.go 
 
package main 
 
func addInt(numbers ...int) int { 
    sum := 0 
    for _, num := range numbers { 
        sum += num 
    } 
    return sum 
} 

addInt_test.go

You might have also noticed in TestSimpleVariadicToSlice that we duplicated a lot of logic, while the only varying factor was the input and expected values. One style of testing, known as Table-driven development, defines a table of all the required data to run a test, iterates over the "rows" of the table and runs tests against them.

Let's look at the tests we will be testing against no arguments and variadic arguments:

// addInt_test.go 
 
package main 
 
import ( 
    "testing" 
) 
 
func TestAddInt(t *testing.T) { 
    testCases := []struct { 
        Name     string 
        Values   []int 
        Expected int 
    }{ 
        {"addInt() -> 0", []int{}, 0}, 
        {"addInt([]int{10, 20, 100}) -> 130", []int{10, 20, 100}, 130}, 
    } 
 
    for _, tc := range testCases { 
        t.Run(tc.Name, func(t *testing.T) { 
            sum := addInt(tc.Values...) 
            if sum != tc.Expected { 
                t.Errorf("%d != %d", sum, tc.Expected) 
            } else { 
                t.Logf("%d == %d", sum, tc.Expected) 
            } 
        }) 
    } 
} 

Running tests in addInt_test.go

Let's now run the tests in this file, and we are expecting each of the row in the testCases table, which we ran, to be treated as a separate test:

$ go test -v ./{addInt.go,addInt_test.go}                           
=== RUN   TestAddInt                       
=== RUN   TestAddInt/addInt()_->_0         
=== RUN   TestAddInt/addInt([]int{10,_20,_100})_->_130                                
--- PASS: TestAddInt (0.00s)               
    --- PASS: TestAddInt/addInt()_->_0 (0.00s)                                        
        addInt_test.go:23: 0 == 0          
    --- PASS: TestAddInt/addInt([]int{10,_20,_100})_->_130 (0.00s)                    
        addInt_test.go:23: 130 == 130      
PASS                                       
ok      command-line-arguments  0.001s       

nil_test.go

We can also create tests that are not specific to any particular source file; the only criteria is that the filename needs to have the <text>_test.go form. The tests in nil_test.go elucidate on some useful features of the language which the developer might find useful while writing tests. They are as follows:

  • httptest.NewServer: Imagine the case where we have to test our code against a server that sends back some data. Starting and coordinating a full blown server to access some data is hard. The http.NewServer solves this issue for us.
  • t.Helper: If we use the same logic to pass or fail a lot of testCases, it would make sense to segregate this logic into a separate function. However, this would skew the test run call stack. We can see this by commenting t.Helper() in the tests and rerunning go test.

We can also format our command-line output to print pretty results. We will show a simple example of adding a tick mark for passed cases and cross mark for failed cases.

In the test, we will run a test server, make GET requests on it, and then test the expected output versus actual output:

// nil_test.go 
 
package main 
 
import ( 
    "fmt" 
    "io/ioutil" 
    "net/http" 
    "net/http/httptest" 
    "testing" 
) 
 
const passMark = "\u2713" 
const failMark = "\u2717" 
 
func assertResponseEqual(t *testing.T, expected string, actual string) { 
    t.Helper() // comment this line to see tests fail due to 'if expected != actual' 
    if expected != actual { 
        t.Errorf("%s != %s %s", expected, actual, failMark) 
    } else { 
        t.Logf("%s == %s %s", expected, actual, passMark) 
    } 
} 
 
func TestServer(t *testing.T) { 
    testServer := httptest.NewServer( 
        http.HandlerFunc( 
            func(w http.ResponseWriter, r *http.Request) { 
                path := r.RequestURI 
                if path == "/1" { 
                    w.Write([]byte("Got 1.")) 
                } else { 
                    w.Write([]byte("Got None.")) 
                } 
            })) 
    defer testServer.Close() 
 
    for _, testCase := range []struct { 
        Name     string 
        Path     string 
        Expected string 
    }{ 
        {"Request correct URL", "/1", "Got 1."}, 
        {"Request incorrect URL", "/12345", "Got None."}, 
    } { 
        t.Run(testCase.Name, func(t *testing.T) { 
            res, err := http.Get(testServer.URL + testCase.Path) 
            if err != nil { 
                t.Fatal(err) 
            } 
 
            actual, err := ioutil.ReadAll(res.Body) 
            res.Body.Close() 
            if err != nil { 
                t.Fatal(err) 
            } 
            assertResponseEqual(t, testCase.Expected, fmt.Sprintf("%s", actual)) 
        }) 
    } 
t.Run("Fail for no reason", func(t *testing.T) {
assertResponseEqual(t, "+", "-")
})
}

Running tests in nil_test.go

We run three tests, where two test cases will pass and one will fail. This way we can see the tick mark and cross mark in action:

$ go test -v ./nil_test.go                                          
=== RUN   TestServer                       
=== RUN   TestServer/Request_correct_URL   
=== RUN   TestServer/Request_incorrect_URL 
=== RUN   TestServer/Fail_for_no_reason    
--- FAIL: TestServer (0.00s)               
  --- PASS: TestServer/Request_correct_URL (0.00s)                                  
        nil_test.go:55: Got 1. == Got 1.  
  --- PASS: TestServer/Request_incorrect_URL (0.00s)                                
        nil_test.go:55: Got None. == Got None. 
--- FAIL: TestServer/Fail_for_no_reason (0.00s)
nil_test.go:59: + != -
FAIL
exit status 1
FAIL command-line-arguments 0.003s