[{"id":"ch01lvl1sec01","contentType":"text","title":"Getting Started with Go and Unix Systems Programming","content":"

Getting Started with Go and Unix Systems Programming

An operating system is the kind of software that allows you to communicate with the hardware, which means that you cannot use your hardware without an operating system. Unix is an operating system with many variants that have many things in common including their programming interface.

The Unix operating system was mainly programmed in C and not entirely in the assembly language, which makes it portable to other computer architectures without having to rewrite everything from scratch. It is important to understand that even if you are developing a Go program on a Unix machine, at the end of the day, your code will be translated to C functions and system calls because this is the only way to directly communicate with the Unix kernel. The main benefits you get from writing Go code instead of C code are smaller programs with less silly bugs. You will learn more about this in Chapter 3, Advanced Go Features.

As this book will use Go, you will need to have a version of Go installed on your Unix machine. The good news is that there is a port of the Go programming language for almost all modern Unix systems including macOS, Linux, and FreeBSD. There is also a Windows port of Go, but this book will not deal with Microsoft Windows.

Although there is a good chance that your Unix variant has a package for Go, you can also get Go from https://golang.org/dl/.

In this chapter, you will learn the following topics:

Systems programming
The advantages and disadvantages of Go
The states of a Unix process
Two Go tools: gofmt and godoc
The features of the latest Go version (1.8)

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1","metaDescription":"","titleSlug":"getting-started-with-go-and-unix-systems-programming"},{"id":"ch01lvl1sec02","contentType":"text","title":"The structure of the book","content":"

The structure of the book

This book has three parts. The first part, which includes this chapter, is about Go and the Go features that can be handy when developing systems software: this does not mean that you should use all of them when developing your programs. The second part is all about programming with files, directories, and processes, which is the most common type of systems software. The third part explores goroutines, web applications, and network programming in Go, which is the most advanced type of systems software. The good thing is that you do not need to read the third part of the book right away.

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1/ch01lvl1sec02/the-structure-of-the-book","metaDescription":"","titleSlug":"the-structure-of-the-book"},{"id":"ch01lvl1sec03","contentType":"text","title":"What is systems programming?","content":"

What is systems programming?

Systems programming is a special area of programming on Unix machines. Note that systems programming is not limited to Unix machines: it is just that this book deals with the Unix operating system only. Most commands that have to do with system administration tasks, such as disk formatting, network interface configuration, module loading, and kernel performance tracking, are implemented using the techniques of systems programming. Additionally, the /etc directory, which can be found on all Unix systems, contains plain text files that deal with the configuration of a Unix machine and its services and are also manipulated using systems software.

You can group the various areas of systems software and related system calls in the following sets:

File I/O: This area deals with file reading and writing operations, which is the most important task of an operating system. File input and output must be fast and efficient, and above all, reliable.
Advanced file I/O: Apart from the basic input and output system calls, there are also more advanced ways to read or write to a file including asynchronous I/O and non-blocking I/O.
System files and configuration: This group of system software includes functions that allow you to handle system files, such as /etc/passwd, and get system specific information, such as system time and DNS configuration.
Files and directories: This cluster includes functions and system calls that allow the programmer to create and delete directories and get information such as the owner and the permissions of a file or a directory.
Process control: This group of software allows you to create and interact with Unix processes.
Threads: When a process has multiple threads, it can perform multiple tasks. However, threads must be created, terminated, and synchronized, which is the purpose of this collection of functions and system calls.
Server processes: This set includes techniques that allow you to develop server processes, which are processes that get executed in the background without the need for an active terminal. Go is not that good at writing server processes in the traditional Unix way: but let me explain this a little more. Unix servers such as Apache use fork(2) to create one or more child processes (this process is called forking and refers to cloning the parent process into a child process) and continue executing the same executable from the same point, and most importantly, sharing memory. Although Go does not offer an equivalent to the fork(2) function, this is not an issue because you can use goroutines to cover most of the uses of fork(2).
Interprocess communication: This set of functions allows processes that run on the same Unix machine to communicate with each other using features such as pipes, FIFOs, message queues, semaphores, and shared memory.
Signal processing: Signals offer processes a way of handling asynchronous events, which can be very handy. Almost all server processes have extra code that allows them to handle Unix signals using the system calls of this group.
Network programming: This is the art of developing applications that work over computer networks with the help of TCP/IP and is not systems programming per se. However, most TCP/IP servers and clients are dealing with system resources, users, files, and directories. So, most of the time, you cannot create network applications without doing some kind of systems programming.

The challenging thing with systems programming is that you cannot afford to have an incomplete program; you can either have a fully working, secure program that can be used on a production system or nothing at all. This mainly happens because you cannot trust end users and hackers. The key difficulty in systems programming is the fact that an erroneous system call can make your Unix machine misbehave or, even worse, crash!

Most security issues on Unix systems usually come from wrongly implemented systems software because bugs in systems software can compromise the security of an entire system. The worst part is that this can happen many years after using a certain piece of software.

When writing systems software, you should take good care of both error messages and warnings because they are the friends that help you understand what is going on and why your program did not behave as expected. Putting it simply, there is a big difference between the File not found and Not enough permissions to read file error messages.

Back when Unix was first introduced, the only way to write systems software was using C; nowadays, you can program systems software using programming languages including Go, which will be the subject of this book.

You should understand that the two main benefits you get from using a programming language other than C for developing systems software are as follows:

Using a modern programming language along with its tools
Simplicity, as you usually have to write, debug, and maintain less code

Apart from Go, other good candidates for developing system tools are Python, Perl, Rust, and Ruby.

Learning systems programming

The only way you can learn systems programming is by developing your own utilities using this book as a reference and a tutorial. At first, you will make a large amount of ridiculous mistakes, but as you get better, you will make a smaller amount of much more clever and hard to debug mistakes! However, it is fine to try new things when learning. In fact, it is necessary to try new things and fail because this means that you are really learning something new. Just make sure that you do not use a production web server for learning systems programming.

If you have difficulties finding out what to develop, you can start by creating your own versions of some of the existing Unix command line utilities such as ls(1), mkdir(1), ln(1), wc(1), and which(1). You do not have to create a fully featured version of each one of them with support for all command-line options; what is important is to develop a stable and secure version that implements the main functionality and works without problems.

The best book that can teach you Unix systems programming in C is Advanced Unix Programming in the Unix Environment by W. Richard Stevens. Its third edition is available now, but all its editions are useful and contain a plethora of valuable details.

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1/ch01lvl1sec03/what-is-systems-programming","metaDescription":"","titleSlug":"what-is-systems-programming"},{"id":"ch01lvl1sec04","contentType":"text","title":"About Go","content":"

About Go

Go is a modern generic purpose open source programming language that was officially announced at the end of 2009. It began as an internal Google project and has been inspired by many other programming languages including C, Pascal, Alef, and Oberon. Its spiritual fathers are Robert Griesemer, Ken Thomson, and Rob Pike, who designed Go as a language for professional programmers who want to build reliable and robust software. Apart from its syntax and standard functions, Go comes with a pretty rich standard library.

At the time of writing this book, the latest stable Go version is 1.8, which includes some handy new features including the following: feel free to skip this if you have not used Go before:

New conversion rules exist that allow you to easily convert between types that are almost equal provided that some criteria are met. You can fix the import paths of the golang.org/x/net/name form to just the name of the Go source file using the go tool command without having to open the source files yourselves.
The operation of the tool is stricter in some cases and looser in cases that used to generate false positives.
There is now a default value for GOPATH Environment Variables when GOPATH is undefined. For Unix systems, the default value is $HOME/go.
There are various improvements to the Go runtime that speed up Go.
There is a sort.slice() function that allows you to sort a slice by providing a comparator callback instead of implementing sort.Interface.
There is now a Shutdown method to http.Server.
There exist various small changes to the database/sql package that give the developer more control over queries.
You can create bugs using the go bug command.

Getting ready for Go

You can easily find your version of Go using this command:

$ go version
go version go1.7.5 darwin/amd64

The previous output is from a macOS machine hence the darwin string. A Linux machine would give the following kind of output:

$ go version
go version go1.3.3 linux/amd64

You will learn more about go tool, which you will use all the time, in the next chapters.

As I can imagine, you must be impatient to see some Go code; so here is the Go version of the famous Hello World program:

package main 
 
import "fmt" 
 
// This is a demonstrative comment! 
func main() { 
   fmt.Println("Hello World!") 
}

If you are familiar with C or C++, you will find Go code pretty easy to understand. Each file that contains Go code begins with a package declaration followed by the needed import declarations. The package declaration shows the package that this file belongs to. Note that semicolons are not required for successfully terminating a Go statement unless you want to put two or more Go statements in the same line.

In Chapter 2, Writing Programs in Go, you will find out how to compile and execute Go code. For now, it is enough to remember that Go source files are stored using the .go file extension: your task is to choose a descriptive filename.

When searching for Go-related information, use Golang or golang as the keyword for the Go programming language because the word Go can be found almost everywhere in the English language and it will not help your search!

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1/ch01lvl1sec04/about-go","metaDescription":"","titleSlug":"about-go"},{"id":"ch01lvl1sec05","contentType":"text","title":"Two useful Go tools","content":"

Two useful Go tools

The Go distribution comes with a plethora of tools that can make your life as a programmer easier. The two most useful of them are gofmt and godoc.

Note that go tool itself can also invoke various tools: you can see a list of them by executing go tool.

The gofmt utility formats Go programs in a given way, which is really important when different people are going to work with the same code for a big project. You can find more information about gofmt at https://golang.org/cmd/gofmt/.

The following is a poorly formatted version of the hw.go program that is hard to read and understand:

$ cat unglyHW.go
package main
    
import
    "fmt"
    
// This is a demonstrative comment!
        func main() {
  fmt.Println("Hello World!")
    
}

Processing the previous code, which is saved as unglyHW.go with gofmt, generates the following easy to read and comprehend output:

$ gofmt unglyHW.go
package main
    
import "fmt"
    
// This is a demonstrative comment!
func main() {
      fmt.Println("Hello World!")
    
}

Remembering that the gofmt utility does not automatically save the generated output is important, which means that you should either use the -w option followed by a valid filename or redirect the output of gofmt to a new file.

The godoc utility allows you to see the documentation of existing Go packages and functions. You can find more information about godoc at http://godoc.org/golang.org/x/tools/cmd/godoc.

You are going to use godoc a lot as it is a great tool for learning the details of Go functions.

The following screenshot shows the output of the godoc command generated on a Terminal when asked for information about the Println() function of the fmt package:

The output of the godoc command

Another handy feature of godoc is that it can start its own web server and allow you to see its documentation using a web browser:

$ godoc -http=:8080

The following screenshot shows the kind of output you get on a web browser after visiting http://localhost:8080/pkg/ while the previous command is running. You can use any port number you want, provided that it is not already in use:

Using the godoc utility from your web browser

The most important tool for a programmer is the editor they use for writing the source code. When I am on a Mac, I typically use the TextMate editor, but when I am on a different Unix machine, I prefer vi. Choosing an editor is not an easy task because you are going to spend a lot of time with it. However, any text editor will do the job as long as it does not put any control characters inside the source code files. The following screenshot shows the TextMate editor in action:

The TextMate editor showing the look of a some Go code

Advantages and disadvantages of Go

Go is not perfect but it has some very interesting features. The list of the Go strong features includes the following:

Go code is easy to read and easy to understand.
Go wants happy developers because a happy developer writes better code!
The Go compiler prints practical warning and error messages that help you solve the actual problem. Putting it simply, the Go compiler is there to help you, not to make your life difficult!
Go code is portable.
Go is a modern programming language.
Go has support for procedural, concurrent, and distributed programming.
Go supports Garbage Collection (GC) so you do not have to deal with memory allocation and deallocation. However, GC might slow down your programs a little.
Go does not have a preprocessor and does high-speed compilation. Consequently, Go can be used as a scripting language.
Go can build web applications. Building a web application in C is simply not very efficient unless you use a nonstandard external library. Additionally, Go provides programmers with a simple web server for testing purposes.
The standard Go library offers many packages that simplify the work of the programmer. Additionally, the methods found in the standard Go library are tested and debugged in advance, which means that most of the time they contain no bugs.
Go uses static linking by default, which means that the produced binary files can be easily transferred to other machines with the same OS. Consequently, the developer does not need to worry about libraries, dependencies, and different library versions.
You will not need a GUI for developing, debugging, and testing Go applications as Go can be used from the command line.
Go supports Unicode. This means that you do not need any extra code to print characters from multiple human languages.
Go keeps concepts orthogonal because a few orthogonal features work better than many overlapping ones.

The list of Go disadvantages includes the following:

Well, Go is not C, which means that you or your team should learn a new programming language to develop systems software.
Go does not have direct support for object-oriented programming, which can be a problem for programmers that are used to writing code in an object-oriented manner. Nevertheless, you can use composition in Go to mimic inheritance.
Back when Unix was first introduced, C was the only programming language for writing systems software. Nowadays, you can also use Rust, C++, and Swift for writing systems software, which means that not everybody will be using Go.
C is still faster than any other programming language for systems programming mainly because Unix is written in C.

Despite the advantages or the disadvantages of a programming language, you have the final word on whether you like it or not. The important thing is that you choose a programming language that you like and can do the job you want! Personally, I do not like C++ despite the fact that it is a very capable programming language and I have written an FTP client in C++! Additionally, I never liked Java. There is no right or wrong thing in personal tastes so do not feel guilty about your choices.

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1/ch01lvl1sec05/two-useful-go-tools","metaDescription":"","titleSlug":"two-useful-go-tools"},{"id":"ch01lvl1sec06","contentType":"text","title":"The various states of a Unix process","content":"

The various states of a Unix process

Strictly speaking, a process is an execution environment that contains instructions, user-data and system-data parts, and other kinds of resources that are obtained during runtime. A program is a file that contains instructions and data, which are used for initializing the instruction and user-data parts of a process.

Back when the Unix operating system was first introduced, computers had single CPUs without multiple cores and a small amount of RAM. However, Unix was a multiuser and multitasking operating system. In order to actually be a multiuser and do multitasking, it had to be able to run each individual process sporadically, which means that a process should have multiple states. The following figure shows the possible states of a process as well as the right path to go from one state to another:

The states of a Unix process

There are three categories of processes: user processes, Kernel processes, and Daemon processes:

User processes run in user space and usually have no special access rights
Kernel processes are being executed in kernel space only and can fully access all kernel data structures
Daemon processes are programs that can be found in the user space and run in the background without the need for a Terminal

Realizing that you cannot control the state of a process is really important, as this is the job of the scheduler of the operating system that runs in the kernel. Putting it simply, you cannot tell when the state of a process is going to change or when the process is going to go into the running state, so your code cannot count on any such assumptions!

The C way for creating new processes involves the calling of the fork() system call. The return value of fork() allows the programmer to differentiate between the parent and child processes. However, Go does not support a similar functionality.

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1/ch01lvl1sec06/the-various-states-of-a-unix-process","metaDescription":"","titleSlug":"the-various-states-of-a-unix-process"},{"id":"ch01lvl1sec07","contentType":"text","title":"Exercises","content":"

Exercises

Visit the Go website: https://golang.org/.
Install Go on your system and find out its version.
Type the code of the Hello World program on your own and save it to a file.
If you are on a Mac, download TextMate from http://macromates.com/.
If you are on a Mac, download the TextWrangler editor from http://www.barebones.com/products/TextWrangler/ and try it.
Try to learn vi or Emacs on your own if you are not already familiar with another Unix text editor.
Look at any Go code you can find and try to make small changes to it.

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1/ch01lvl1sec07/exercises","metaDescription":"","titleSlug":"exercises"},{"id":"ch01lvl1sec08","contentType":"text","title":"Summary","content":"

Summary

In this chapter, you learned how to get Go on your computer, the features of the latest Go version, the advantages and disadvantages of Go, and the gofmt and godoc Go tools, as well as some important things about the Unix operating system.

The next chapter will not only tell you how to compile your Go code but it will also discuss other important Go topics such as reading and using command-line arguments, environment variables, writing functions, data structures, interfaces, getting user input, and printing output.

","url":"https://subscription.packtpub.com/book/programming/9781787125643/1/ch01lvl1sec08/summary","metaDescription":"","titleSlug":"summary"}]

Go Systems Programming

Go Systems Programming

Overview of this book

Related Content you might be interested in

Current Title:

Go Systems Programming

Mastering Go

Mastering Go.

Mastering Go – Third Edition