Book Image

Security with Go

By : John Daniel Leon
Book Image

Security with Go

By: John Daniel Leon

Overview of this book

Go is becoming more and more popular as a language for security experts. Its wide use in server and cloud environments, its speed and ease of use, and its evident capabilities for data analysis, have made it a prime choice for developers who need to think about security. Security with Go is the first Golang security book, and it is useful for both blue team and red team applications. With this book, you will learn how to write secure software, monitor your systems, secure your data, attack systems, and extract information. Defensive topics include cryptography, forensics, packet capturing, and building secure web applications. Offensive topics include brute force, port scanning, packet injection, web scraping, social engineering, and post exploitation techniques.
Table of Contents (15 chapters)

About Go

Go is an open source programming language that was created by Google and distributed under a BSD-style license. A BSD license allows anyone to use Go free of charge, as long as the copyright notice is retained and the Google name is not used for endorsement or promotion. Go is heavily influenced by C, but has simpler syntax, and better memory safety and garbage collection. Sometimes, Go is described as a modern-day C++. I think that is too much of a simplification, but Go is definitely a simple but modern language.

Go language design

The original goal of Go was to create a new language that is simple, reliable, and efficient. As mentioned, Go is heavily influenced by C programming language. The language itself is very simple, with only 25 keywords. It was built to integrate well with IDEs, but not to be dependent on them. In my experience, anyone who has tried Go has found it very user-friendly with a short learning curve.

One of the main goals of Go was to deal with some of the negative aspects of C++ and Java code, while retaining the performance. The language needed to be simple and consistent to manage very large development teams.

Variables are statically typed, and applications are compiled quickly to statically linked binaries. Having a single statically linked binary makes it very easy to create lightweight containers. The final applications run fast as well, running close to C++ and Java performance and much faster than interpreted languages such as Python. There are pointers, but there is no pointer arithmetic allowed. Go does not tout itself as an object-oriented programming language, and it does not formally have classes in the traditional sense; however, it does contain a number of mechanisms that closely resemble an object-oriented programming language. This is discussed in more depth in the following chapter. Interfaces are used heavily, and composition is the equivalent of inheritance.

Go has many interesting features. One feature that stands out is the built-in concurrency. Just put the word "go" before any function call, and it will spawn a lightweight thread to execute the function. Another feature that is quite important is the dependency management, which is very efficient. The dependency management is part of the reason Go compiles incredibly fast. It does not re-include the same header files multiple times, the way C++ does. Go also has built-in memory safety, and a garbage collector handles clean-up of unused memory. The standard library in Go is pretty impressive too. It is modern and contains networking, HTTP, TLS, XML, JSON, database, image manipulation, and cryptography packages. Go also supports Unicode, allowing all kinds of characters to be used in source code.

The Go toolchain is central to the ecosystem. It provides tools to download and install remote dependencies, run unit tests and benchmarks, generate code, and format code according to the Go formatting standards. It also includes the compiler, linker, and assembler, which compile very quickly and also allow for easy cross-compiling by simply changing the GOOS and GOARCH environment variables.

Some features were excluded from the Go language. Generics, inheritance, assertions, exceptions, pointer arithmetic, and implicit type conversions were all left out of Go. Many features were omitted intentionally, namely generics, assertions, and pointer arithmetic. The authors left out some features because they wanted to maintain performance, keep the language specification as simple as possible, or they could not agree on the best way to implement, or because a feature was too controversial. Inheritance was also left out intentionally in favor of using interfaces and composition instead. Some other features, such as generics, were left out because there was too much debate concerning their proper implementation, but they may show up in Go 2.0. The authors recognized that it is much easier to add a feature to a language than to take one away.

The History of Go

Go is a relatively young language, with its inception in 2007 and open sourcing in 2009. It started at Google as a 20% project with Robert Griesemer, Rob Pike, and Ken Thompson. A 20% project means that the project's developers spent 20% of their time working on it as an experimental side project. Go 1.0 was officially released in March 2012. It was planned from the beginning to be an open source language. Until Go 1.5, the compiler, linker, and assembler were written in C. After version 1.5, everything was written in Go.

Google initially launched Go for Linux and macOS, and the community drove the effort for other platforms, namely Windows, FreeBSD, OpenBSD, NetBSD, and Solaris. It has even been ported to IBM z Systems mainframes. Bill O'Farrell of IBM gave a talk at GopherCon 2016 in Denver titled Porting Go to the IBM z Architecture (https://www.youtube.com/watch?v=z0f4Wgi94eo).

Google is known to use Python, Java, and C++. It is understandable why they chose those languages too. They each fill certain roles and have their own strengths and weaknesses. Go was an attempt to create a new language that fits the needs of Google. They needed software that could perform extremely well under heavy loads, support concurrency, and that was easy to read and write, and compile quickly.

The triggering event for starting the Go project was dealing with a massive C++ code base that took hours to build because of the way C++ handles dependencies and re-includes headers (https://www.youtube.com/watch?v=bj9T2c2Xk_s (37:15)). This is why one of Go's main goals was to compile quickly. Go helped turn hours of compile time to seconds because it handles dependencies much more efficiently than C++.

Discussions for Go 2.0 have begun, but they are still in the conceptual stages. There is no timeline for a release, and there is no rush to release a new major version.

Adoption and community

Go is still a young language, but it has seen growing adoption rates and has continued to increase in popularity. Go was the TIOBE Language of the year in 2009 and 2016:

Source: https://www.tiobe.com/tiobe-index/go/

One of the expectations expressed by the Go team was the anticipation that Go would draw a lot of C/C++ and Java developers, but they were surprised when a large number of the users came from scripting languages such as Python and Ruby. Others, like myself, found Go to be a natural complement to Python, a great language. However, which language do you go to when you need something much more powerful? Some large companies have demonstrated that Go is stable for large-scale production use, including Google, Dropbox, Netflix, Uber, and SoundCloud.

The first Go conference, named GopherCon, was held in 2014. Since then, GopherCon has been held every year. Read more about GopherCon at https://gophercon.com. I had the privilege of speaking at the 2016 GopherCon about packet capturing and had a great experience (https://www.youtube.com/watch?v=APDnbmTKjgM).

Common criticisms about Go

There are a few criticisms that show up repeatedly in the community. Probably the most notorious and most-discussed criticism is the lack of generics. This leads to repeated code to handle different data types. Interfaces can be used to some extent to alleviate this problem. We might see generics in a future release, as the authors have shown openness to generics, but they did not rush through an important design decision.

The next criticism often heard is the lack of exceptions. The developer must explicitly handle or ignore each error. Personally, I found this to be a refreshing change. It's not really that much more work, and you have full control over the code flow. Sometimes with exceptions you are not positive where it will get caught as it bubbles up. With Go, you can easily follow the error-handling code.

Go has a garbage collector that handles memory cleanup. The garbage collector has been upgraded over time and continues to improve. The garbage collector does have a minor performance impact, but saves the developer a lot of thinking and worrying. Go was originally described as a systems programming language, and the lack of control over the memory was restrictive for very low-level applications. Since then, they have pivoted away from calling Go a systems programming language. If you need low-level control over memory, then you will have to write portions of code in C.

The Go toolchain

The go executable is the main application of the Go toolchain. You can pass a command to go, and it will take the appropriate action. The toolchain has tools to run, compile, format source code, download dependencies, and more. Let's look at the full list, which is obtained as an output from the go help command or just go by itself:

  • build: This compiles packages and dependencies
  • clean: This removes object files
  • doc: This shows documentation for a package or symbol
  • env: This prints Go environment information
  • generate: This is the code generator
  • fix: This upgrades Go code when a new version is released
  • fmt: This runs gofmt on package sources
  • get: This downloads and installs packages and dependencies
  • help: This provides more help on a specific topic
  • install: This compiles and installs packages and dependencies
  • list: This lists packages
  • run: This compiles and runs Go programs
  • test: This runs unit tests and benchmarks
  • vet: This examines source code for bugs
  • version: This shows the Go version

More information about these commands is available at https://golang.org/cmd/.

Go mascot

Everyone knows that the best swords have names, and the best programming languages have mascots. Go's mascot is the gopher. The gopher has no name. It has a jelly bean shaped body, microscopic limbs, gigantic eyes, and two teeth. It was designed by Renee French, and its copyright comes under the Creative Commons Attribution 3.0 license. This means that you can play with the images, but you must give credit to their creator, Renee French, wherever they are used.

Renee French gave a talk at GopherCon 2016 in Denver, entitled The Go Gopher: A Character Study, explaining how the gopher came to be, the various mediums and forms it has taken, and the tips on drawing it in various situations (https://www.youtube.com/watch?v=4rw_B4yY69k).

You can generate a custom gopher avatar at https://gopherize.me/ and read more about the Go gopher at https://blog.golang.org/gopher.

Learning Go

If you have never used Go before, have no fear. It has a gentle learning curve and is simple enough to learn in just a day or two. The best place to start is https://tour.golang.org/. This is a basic tour of the Go programming language. If you have already gone through the tour, then you should already have the foundation to make it through this book just fine. If you are working through this book and have not taken the tour, you may come across a concept you are not familiar with that is not explained here. The tour is a good place to learn and practice.

Since there are only 25 reserved keywords in the language specification, it is short enough to be understood by "mortals". You can read more about the specs at https://golang.org/ref/spec.

You must be already be familiar with most of these keywords. These are: if, else, goto, for, import, return, var, continue, break, range, type, func, interface, package, const, map, struct, select, case, switch, go, defer, chan, fallthrough, and default.

The tour will help you learn the keywords, syntaxes, and basics of the data structures. The playground in the tour lets you practice writing and running code in the browser.

Why use Go?

There are several aspects that appeal to me about Go. Concurrency, speed, and simplicity are the most important things to me. The language is very simple and easy to learn. There are no try, catch, and exception flows. Though several people cite the tedious error handling as a criticism, I find it refreshing to have a simple language that does not hide a lot of magic behind the scenes and does exactly what it says. The go fmt tool standardizes formatting, making it easy to read code from others, and eliminates the burden of defining your own standard.

Go provides a feeling of scalability and reliability and is actually an enjoyable experience. Before Go, the primary option for fast, compiled code was C++, and it was no simple task to manage the header files and build processes for different platforms. C++ has become a very complicated language over the years and is not nearly as approachable as Go for most people.