[{"id":"ch01lvl1sec01","contentType":"text","title":"I/O and File Systems","content":"

I/O and File Systems

In this chapter, the following recipes will be covered:

Using the common I/O interfaces
Using the bytes and strings packages
Working with directories and files
Working with the CSV format
Working with temporary files
Working with text/template and HTML/templates

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1","metaDescription":"","titleSlug":"i-o-and-file-systems"},{"id":"ch01lvl1sec02","contentType":"text","title":"Introduction","content":"

Introduction

Go provides excellent support for both basic and complex I/O. The recipes in this chapter will explore common Go interfaces to deal with I/O and show how to make use of them. The Go standard library frequently uses these interfaces, and these interfaces will be used by recipes throughout the book.

You'll learn how to work with data in memory and in the form of streams. You'll see examples of working with files and directories and of working with the CSV format. The temporary files recipe discusses a mechanism to work with files without the overhead of dealing with name collision and more. Lastly, we'll explore Go standard templates for both plain text and HTML.

These recipes should lay the foundation for the use of interfaces to represent and modify data and should help you think about data in an abstract and flexible way.

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1/ch01lvl1sec02/introduction","metaDescription":"","titleSlug":"introduction"},{"id":"ch01lvl1sec03","contentType":"text","title":"Using the common I/O interfaces","content":"

Using the common I/O interfaces

Go provides a number of I/O interfaces used throughout the standard library. It is a best practice to make use of these interfaces wherever possible rather than passing structs or other types directly. Two powerful interfaces we explore in this recipe are the io.Reader and io.Writer interfaces. These interfaces are used throughout the standard library and understanding how to use them will make you a better Go developer.

The Reader and Writer interfaces look like this:

type Reader interface {
        Read(p []byte) (n int, err error)
}

type Writer interface {
        Write(p []byte) (n int, err error)
}

Go also makes it easy to combine interfaces. For example, take a look at the following code:

type Seeker interface {
        Seek(offset int64, whence int) (int64, error)
}

type ReadSeeker interface {
        Reader
        Seeker
}

The recipe will also explore an io function called Pipe():

func Pipe() (*PipeReader, *PipeWriter)

The remainder of this book will make use of these interfaces.

Getting ready

Configure your environment according to these steps:

Download and install Go on your operating system at https://golang.org/doc/install and configure your GOPATH environment variable.

Open a terminal/console application, navigate to your GOPATH/src directory, and create a project directory such as $GOPATH/src/github.com/yourusername/customrepo.

All code will be run and modified from this directory.

Optionally, install the latest tested version of the code using the following command:

      go get github.com/agtorre/go-cookbook/

How to do it...

These steps cover writing and running your application:

From your terminal/console application, create a new directory called chapter1/interfaces.
Navigate to that directory.

Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/interfaces, or use this as an exercise to write some of your own code.

Create a file called interfaces.go with the following contents:

        package interfaces

        import (
                "fmt"
                "io"
                "os"
        )

        // Copy copies data from in to out first directly,
        // then using a buffer. It also writes to stdout
        func Copy(in io.ReadSeeker, out io.Writer) error {
                // we write to out, but also Stdout
                w := io.MultiWriter(out, os.Stdout)
        
                // a standard copy, this can be dangerous if there's a 
                // lot of data in in
                if _, err := io.Copy(w, in); err != nil {
                    return err
                }
        
                in.Seek(0, 0)

                // buffered write using 64 byte chunks
                buf := make([]byte, 64)
                if _, err := io.CopyBuffer(w, in, buf); err != nil {
                    return err
                }

                // lets print a new line
                fmt.Println()
        
                return nil
        }

Create a file called pipes.go with the following contents:

        package interfaces

        import (
                "io"
                "os"
        )
        
        // PipeExample helps give some more examples of using io  
        //interfaces
        func PipeExample() error {
                // the pipe reader and pipe writer implement
                // io.Reader and io.Writer
                r, w := io.Pipe()

                // this needs to be run in a separate go routine
                // as it will block waiting for the reader
                // close at the end for cleanup
                go func() {
                    // for now we'll write something basic,
                    // this could also be used to encode json
                    // base64 encode, etc.
                    w.Write([]byte("testn"))
                    w.Close()
                }()
        
                if _, err := io.Copy(os.Stdout, r); err != nil {
                    return err
                }
                return nil
        }

Create a new directory named example.
Navigate to example.

Create a main.go file with the following contents and ensure that you modify the interfaces imported to use the path you set up in step 2:

        package main

        import (
             "bytes"
             "fmt"
        
             "github.com/agtorre/go-cookbook/chapter1/interfaces"
        )
        
        func main() {
                in := bytes.NewReader([]byte("example"))
                out := &bytes.Buffer{}
                fmt.Print("stdout on Copy = ")
                if err := interfaces.Copy(in, out); err != nil {
                        panic(err)
                }
        
                fmt.Println("out bytes buffer =", out.String())
        
                fmt.Print("stdout on PipeExample = ")
                if err := interfaces.PipeExample(); err != nil {
                        panic(err)
                }
        }

Run go run main.go.
You may also run these:

      go build 
      ./example

You should see the following output:

        $ go run main.go
        stdout on Copy = exampleexample
        out bytes buffer = exampleexample
        stdout on PipeExample = test

If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

The Copy() function copies between interfaces and treats them like streams. Thinking of data as streams has a lot of practical uses, especially when working with network traffic or filesystems. The Copy() function also creates a multi-writer that combines two writer streams and writes to them twice using ReadSeeker. If a Reader interface were used instead rather than seeing exampleexample, you would only see example despite copying to the MultiWriter interface twice. There's also an example of a buffered write that you might use if your stream is not fit into the memory.

The PipeReader and PipeWriter structs implement io.Reader and io.Writer interfaces. They're connected, creating an in-memory pipe. The primary purpose of a pipe is to read from a stream while simultaneously writing from the same stream to a different source. In essence, it combines the two streams into a pipe.

Go interfaces are a clean abstraction to wrap data that performs common operations. This is made apparent when doing I/O operations, and so the io package is a great resource for learning about interface composition. The pipe package is often underused but provides great flexibility with thread-safety when linking input and output streams.

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1/ch01lvl1sec03/using-the-common-i-o-interfaces","metaDescription":"","titleSlug":"using-the-common-i-o-interfaces"},{"id":"ch01lvl1sec04","contentType":"text","title":"Using the bytes and strings packages","content":"

Using the bytes and strings packages

The bytes and string packages have a number of useful helpers to work with and convert between strings and byte types. They allow the creation of buffers that work with a number of common I/O interfaces.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

From your terminal/console application, create a new directory called chapter1/bytestrings.
Navigate to this directory.

Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/bytesstrings, or use this as an exercise to write some of your own code!
Create a file called buffer.go with the following contents:

        package bytestrings
        
        import (
                "bytes"
                "io"
                "io/ioutil"
        )
        
        // Buffer demonstrates some tricks for initializing bytes    
        //Buffers
        // These buffers implement an io.Reader interface
        func Buffer(rawString string) *bytes.Buffer {
        
                // we'll start with a string encoded into raw bytes
                rawBytes := []byte(rawString)
        
                // there are a number of ways to create a buffer from 
                // the raw bytes or from the original string
                var b = new(bytes.Buffer)
                b.Write(rawBytes)
        
                // alternatively
                b = bytes.NewBuffer(rawBytes)
        
                // and avoiding the intial byte array altogether
                b = bytes.NewBufferString(rawString)
        
                return b
        }
        
        // ToString is an example of taking an io.Reader and consuming 
        // it all, then returning a string
        func toString(r io.Reader) (string, error) {
                b, err := ioutil.ReadAll(r)
                if err != nil {
                    return "", err
                }
                return string(b), nil
        }

Create a file called bytes.go with the following contents:

        package bytestrings
        
        import (
                "bufio"
                "bytes"
                "fmt"
        )
        
        // WorkWithBuffer will make use of the buffer created by the
        // Buffer function
        func WorkWithBuffer() error {
                rawString := "it's easy to encode unicode into a byte 
                              array"

                b := Buffer(rawString)

                // we can quickly convert a buffer back into byes with
                // b.Bytes() or a string with b.String()
                fmt.Println(b.String())
        
                // because this is an io Reader we can make use of  
                // generic io reader functions such as
                s, err := toString(b)
                if err != nil {
                    return err
                }
                fmt.Println(s)

                // we can also take our bytes and create a bytes reader
                // these readers implement io.Reader, io.ReaderAt, 
                // io.WriterTo, io.Seeker, io.ByteScanner, and 
                // io.RuneScanner interfaces
                reader := bytes.NewReader([]byte(rawString))
        
                // we can also plug it into a scanner that allows 
                // buffered reading and tokenzation
                scanner := bufio.NewScanner(reader)
                scanner.Split(bufio.ScanWords)
        
                // iterate over all of the scan events
                for scanner.Scan() {
                    fmt.Print(scanner.Text())
                }
        
                return nil
        }

Create a file called string.go with the following contents:

        package bytestrings
        
        import (
                "fmt"
                "io"
                "os"
                "strings"
        )
        
        // SearchString shows a number of methods
        // for searching a string
        func SearchString() {
                s := "this is a test"
        
                // returns true because s contains
                // the word this
                fmt.Println(strings.Contains(s, "this"))
        
                // returns true because s contains the letter a
                // would also match if it contained b or c
                fmt.Println(strings.ContainsAny(s, "abc"))
        
                // returns true because s starts with this
                fmt.Println(strings.HasPrefix(s, "this"))
        
                // returns true because s ends with this
                fmt.Println(strings.HasSuffix(s, "test"))
                }
        
        // ModifyString modifies a string in a number of ways
        func ModifyString() {
                s := "simple string"
        
                // prints [simple string]
                fmt.Println(strings.Split(s, " "))
        
                // prints "Simple String"
                fmt.Println(strings.Title(s))
        
                // prints "simple string"; all trailing and
                // leading white space is removed
                s = " simple string "
                fmt.Println(strings.TrimSpace(s))
        }
        
        // StringReader demonstrates how to create
        // an io.Reader interface quickly with a string
        func StringReader() {
                s := "simple stringn"
                r := strings.NewReader(s)
        
                // prints s on Stdout
                io.Copy(os.Stdout, r)
        }

Create a new directory named example.
Navigate to example.
Create a main.go file with the following contents and ensure that you modify the interfaces imported to use the path you set up in step 2:

        package main
        
        import "github.com/agtorre/go-cookbook/chapter1/bytestrings"
        
        func main() {
                err := bytestrings.WorkWithBuffer()
                if err != nil {
                        panic(err)
                }
        
                // each of these print to stdout
                bytestrings.SearchString()
                bytestrings.ModifyString()
                bytestrings.StringReader() 
        }

Run go run main.go.
You may also run these:

      go build
      ./example

You should see the following output:

        $ go run main.go
        it's easy to encode unicode into a byte array ??
        it's easy to encode unicode into a byte array ??
        it'seasytoencodeunicodeintoabytearray??true
        true
        true
        true
        [simple string]
        Simple String
        simple string
        simple string

If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

The bytes library provides a number of convenience functions when working with data. A buffer, for example, is far more flexible than an array of bytes when working with stream processing libraries or methods. Once you've created a buffer, it can be used to satisfy an io.Reader interface so you can take advantage of ioutil functions to manipulate the data. For steaming applications, you'd probably want to use a buffer and a scanner. The bufio package comes in handy for these cases. Sometimes, using an array or slice is more appropriate for smaller datasets or when you have a lot of memory on your machine.

Go provides a lot of flexibility in converting between interfaces with these basic types--it's relatively simple to convert between strings and bytes. When working with strings, the strings package provides a number of convenience functions to work with, search, and manipulate strings. In some cases, a good regular expression may be appropriate, but most of the time, the strings and strconv packages are sufficient. The strings package allows you to make a string look like a title, split it into an array, or trim whitespace. It also provides a Reader interface of its own that can be used instead of the bytes package reader type.

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1/ch01lvl1sec04/using-the-bytes-and-strings-packages","metaDescription":"","titleSlug":"using-the-bytes-and-strings-packages"},{"id":"ch01lvl1sec05","contentType":"text","title":"Working with directories and files","content":"

Working with directories and files

Working with directories and files can be difficult when you switch between platforms (Windows and Linux, for example). Go provides cross-platform support to work with files and directories in the os and ioutils packages. We've already seen examples of ioutils, but now we'll explore how to use them in another way!

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

From your terminal/console application, create a new directory called chapter1/filedirs.
Navigate to this directory.
Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/filedirs, or use this as an exercise to write some of your own code!
Create a file called dirs.go with the following contents:

        package filedirs
        
        import (
                "errors"
                "io"
                "os"
        )
        
        // Operate manipulates files and directories
        func Operate() error {
                // this 0777 is similar to what you'd see with chown
                // on a command line this will create a director 
                // /tmp/example, you may also use an absolute path 
                // instead of a relative one
                if err := os.Mkdir("example_dir", os.FileMode(0755)); 
                err !=  nil {
                        return err
                }

                // go to the /tmp directory
                if err := os.Chdir("example_dir"); err != nil {
                        return err
                }

                // f is a generic file object
                // it also implements multiple interfaces
                // and can be used as a reader or writer
                // if the correct bits are set when opening
                f, err := os.Create("test.txt")
                if err != nil {
                        return err
                }

                // we write a known-length value to the file and 
                // validate that it wrote correctly
                value := []byte("hellon")
                count, err := f.Write(value)
                if err != nil {
                        return err
                }
                if count != len(value) {
                        return errors.New("incorrect length returned 
                        from write")
                }

                if err := f.Close(); err != nil {
                        return err
                }

                // read the file
                f, err = os.Open("test.txt")
                if err != nil {
                        return err
                }
        
                io.Copy(os.Stdout, f)
        
                if err := f.Close(); err != nil {
                        return err
                }
        
                // go to the /tmp directory
                if err := os.Chdir(".."); err != nil {
                        return err
                }
        
                // cleanup, os.RemoveAll can be dangerous if you
                // point at the wrong directory, use user input,
                // and especially if you run as root
                if err := os.RemoveAll("example_dir"); err != nil {
                        return err
                }
        
                return nil
        }

Create a file called bytes.go with the following contents:

        package filedirs
        
        import (
                "bytes"
                "io"
                "os"
                "strings"
        )
        
        // Capitalizer opens a file, reads the contents,
        // then writes those contents to a second file
                func Capitalizer(f1 *os.File, f2 *os.File) error {
                if _, err := f1.Seek(0, 0); err != nil {
                        return err
                }
        
                var tmp = new(bytes.Buffer)
        
                if _, err := io.Copy(tmp, f1); err != nil {
                        return err
                }

                s := strings.ToUpper(tmp.String())
        
                if _, err := io.Copy(f2, strings.NewReader(s)); err != 
                nil {
                        return err
                }
                return nil
        }
        
        // CapitalizerExample creates two files, writes to one
        //then calls Capitalizer() on both
        func CapitalizerExample() error {
                f1, err := os.Create("file1.txt")
                if err != nil {
                        return err
                }
        
                if _, err := f1.Write([]byte(`this file contains a 
                number of words and new lines`)); err != nil {
                        return err
                }
        
                f2, err := os.Create("file2.txt")
                if err != nil {
                        return err
                }
        
                if err := Capitalizer(f1, f2); err != nil {
                        return err
                }
        
                if err := os.Remove("file1.txt"); err != nil {
                        return err
                }
        
                if err := os.Remove("file2.txt"); err != nil {
                        return err
                }
        
                return nil
        }

Create a new directory named example.
Navigate to example.
Create a main.go file with the following contents and ensure that you modify the filedirs package import to use the path you set up in step 2:

        package main
        
        import "github.com/agtorre/go-cookbook/chapter1/filedirs"
        
        func main() {
                if err := filedirs.Operate(); err != nil {
                        panic(err)
                }
        
                if err := filedirs.CapitalizerExample(); err != nil {
                        panic(err)
                }
        }

Run go run main.go.
You may also run these:

      go build
      ./example

You should see the following output:

        $ go run main.go 
        hello

If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

If you're familiar with files in Unix, the Go os library should feel very familiar. You can do basically all common operations--stat a file to collect attributes, collect a file with different permissions, and create and modify directories and files. We performed a number of manipulations to directories and files and then cleaned up after ourselves.

Working with file objects is very similar to in-memory streams. Files also provide a number of convenience functions directly, such as Chown, Stat, and Truncate. The easiest way to get comfortable with files is to make use of them. In all the previous recipes, we have to be careful to clean up after our programs.

Working with files is a very common operation when building backend applications. Files can be used for configuration, secret keys, as temporary storage, and more. Go wraps OS system calls using the os package and allows the same functions to operate regardless of whether you're using Windows or Unix.

Once your file is opened and stored in a File struct, it can easily be passed into a number of interfaces discussed earlier. All the earlier examples of working with buffers and in-memory data streams can be replaced directly with file objects. This may be useful for things such as writing all logs to stderr and the file at the same time with a single write call.

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1/ch01lvl1sec05/working-with-directories-and-files","metaDescription":"","titleSlug":"working-with-directories-and-files"},{"id":"ch01lvl1sec06","contentType":"text","title":"Working with the CSV format","content":"

Working with the CSV format

CSV is a common format to manipulate data. It's common, for example, to import or export a CSV file into Excel. The Go CSV package operates on data interfaces, and as a result, it's easy to write data to a buffer, stdout, a file, or to a socket. The examples in this section will show some common ways to get data into and out of the CSV format.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

From your terminal/console application, create a new directory called chapter1/csvformat.
Navigate to this directory.
Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/csvformat, or use this as an exercise to write some of your own code!

Create a file called read_csv.go with the following contents:

        package csvformat
        
        import (
                "bytes"
                "encoding/csv"
                "fmt"
                "io"
                "strconv"
        )
        
        // Movie will hold our parsed CSV
        type Movie struct {
                Title string
                Director string
                Year int
        }
        
        // ReadCSV gives shows some examples of processing CSV
        // that is passed in as an io.Reader
        func ReadCSV(b io.Reader) ([]Movie, error) {
        
                r := csv.NewReader(b)
        
                // These are some optional configuration options
                r.Comma = ';'
                r.Comment = '-'
        
                var movies []Movie
        
                // grab and ignore the header for now
                // we may also wanna use this for a dictionary key or
                // some other form of lookup
                _, err := r.Read()
                if err != nil && err != io.EOF {
                        return nil, err
                }
        
                // loop until it's all processed
                for {
                        record, err := r.Read()
                        if err == io.EOF {
                                break
                        } else if err != nil {
                                return nil, err
                        }
        
                        year, err := strconv.ParseInt(record[2], 10, 
                        64)
                        if err != nil {
                                return nil, err
                        }

                        m := Movie{record[0], record[1], int(year)}
                        movies = append(movies, m)
                }
                return movies, nil
        }

        // AddMoviesFromText uses the CSV parser with a string
        func AddMoviesFromText() error {
                // this is an example of us taking a string, converting
                // it into a buffer, and reading it 
                // with the csv package
                in := `
                - first our headers
                movie title;director;year released
        
                - then some data
                Guardians of the Galaxy Vol. 2;James Gunn;2017
                Star Wars: Episode VIII;Rian Johnson;2017
                `
        
                b := bytes.NewBufferString(in)
                m, err := ReadCSV(b)
                if err != nil {
                        return err
                }
                fmt.Printf("%#vn", m)
                return nil
        }

Create a file called write_csv.go with the following contents:

        package csvformat
        
        import (
                "bytes"
                "encoding/csv"
                "io"
                "os"
        )
        
        // A Book has an Author and Title
        type Book struct {
                Author string
                Title string
        }
        
        // Books is a named type for an array of books
        type Books []Book
        
        // ToCSV takes a set of Books and writes to an io.Writer
        // it returns any errors
        func (books *Books) ToCSV(w io.Writer) error {
                n := csv.NewWriter(w)
                err := n.Write([]string{"Author", "Title"})
                if err != nil {
                        return err
                }
                for _, book := range *books {
                        err := n.Write([]string{book.Author, 
                        book.Title})
                        if err != nil {
                                return err
                        }
                }
        
                n.Flush()
                return n.Error()
        }

        // WriteCSVOutput initializes a set of books
        // and writes the to os.Stdout
        func WriteCSVOutput() error {
                b := Books{
                        Book{
                                Author: "F Scott Fitzgerald",
                                Title: "The Great Gatsby",
                        },
                        Book{
                                Author: "J D Salinger",
                                Title: "The Catcher in the Rye",
                        },
                }
        
                return b.ToCSV(os.Stdout)
        }

        // WriteCSVBuffer returns a buffer csv for
        // a set of books
        func WriteCSVBuffer() (*bytes.Buffer, error) {
                b := Books{
                        Book{
                                Author: "F Scott Fitzgerald",
                                Title: "The Great Gatsby",
                        },
                        Book{
                                Author: "J D Salinger",
                                Title: "The Catcher in the Rye",
                        },
                }
        
                w := &bytes.Buffer{}
                err := b.ToCSV(w)
                return w, err
        }

Create a new directory named example.

Navigate to example.
Create a main.go file with the following contents and ensure that you modify the csvformat import to use the path you set up in step 2:

        package main
        
        import (
                "fmt"
        
                "github.com/agtorre/go-cookbook/chapter1/csvformat"
        )
        
        func main() {
                if err := csvformat.AddMoviesFromText(); err != nil {
                        panic(err)
                }
        
                if err := csvformat.WriteCSVOutput(); err != nil {
                        panic(err)
                }
        
                buffer, err := csvformat.WriteCSVBuffer()
                if err != nil {
                        panic(err)
                }
        
                fmt.Println("Buffer = ", buffer.String())
        }

Run go run main.go.
You may also run these:

      go build
      ./example

You should see the following output:

        $ go run main.go 
        []csvformat.Movie{csvformat.Movie{Title:"Guardians of the 
        Galaxy Vol. 2", Director:"James Gunn", Year:2017},         
        csvformat.Movie{Title:"Star Wars: Episode VIII", Director:"Rian 
        Johnson", Year:2017}}
        Author,Title
        F Scott Fitzgerald,The Great Gatsby
        J D Salinger,The Catcher in the Rye
        Buffer = Author,Title
        F Scott Fitzgerald,The Great Gatsby
        J D Salinger,The Catcher in the Rye

If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

In order to explore reading a CSV format, we first represent our data as a struct. It's very useful in Go to format data as a struct, as it makes things such as marshaling and encoding relatively simple. Our read example uses movies as our data type. The function takes an io.Reader interface that holds our CSV data as an input. This could be a file or a buffer. We then use that data to create and populate a Movie struct, including converting the year into an integer. We also add options to the CSV parser to use ; as the separator and - as a comment line.

Next, we explore the same idea, but in reverse. Novels are represented with a title and an author. We initialize an array of novels and then write specific novels in the CSV format to an io.Writer interface. Once again, this can be a file, stdout, or a buffer.

The CSV package is an excellent example of why you'd want to think of data flows in Go as implementing common interfaces. It's easy to change the source and destination of our data with small one-line tweaks, and we can easily manipulate CSV data without using an excessive amount of memory or time. For example, it would be possible to read from a stream of data one record at a time and write to a separate stream in a modified format one record at a time. Doing this would not incur significant memory or processor usage.

Later, when we explore data pipelines and worker pools, you'll see how these ideas can be combined and how to handle these streams in parallel.

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1/ch01lvl1sec06/working-with-the-csv-format","metaDescription":"","titleSlug":"working-with-the-csv-format"},{"id":"ch01lvl1sec07","contentType":"text","title":"Working with temporary files","content":"

Working with temporary files

We've created and made use of files for a number of examples so far. We've also had to manually deal with cleanup, name collision, and more. Temporary files and directories are a quicker, simpler way to handle these cases.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

From your terminal/console application, create a new directory called chapter1/tempfiles.
Navigate to this directory.
Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/tempfiles, or use this as an exercise to write some of your own code!
Create a file called temp_files.go with the following contents:

        package tempfiles
        
        import (
                "fmt"
                "io/ioutil"
                "os"
        )
        
        // WorkWithTemp will give some basic patterns for working
        // with temporary files and directories
        func WorkWithTemp() error {
                // If you need a temporary place to store files with 
                // the same name ie. template1-10.html a temp directory 
                //  is a good way to approach it, the first argument 
                // being blank means it will use create the directory                
                // in the location returned by 
                // os.TempDir()
                t, err := ioutil.TempDir("", "tmp")
                if err != nil {
                        return err
                }
        
                // This will delete everything inside the temp file 
                // when this function exits if you want to do this 
                //  later, be sure to return the directory name to the 
                // calling function
                defer os.RemoveAll(t)
        
                // the directory must exist to create the tempfile
                // created. t is an *os.File object.
                tf, err := ioutil.TempFile(t, "tmp")
                if err != nil {
                        return err
                }
        
                fmt.Println(tf.Name())
        
                // normally we'd delete the temporary file here, but 
                // because we're placing it in a temp directory, it 
                // gets cleaned up by the earlier defer

                return nil
        }

Create a new directory named example.
Navigate to example.
Create a main.go file with the following contents and ensure that you modify the tempfiles imported to use the path you set up in step 2:

        package main
        
        import "github.com/agtorre/go-cookbook/chapter1/tempfiles"
        
        func main() {
                if err := tempfiles.WorkWithTemp(); err != nil {
                        panic(err)
                }
        }

Run go run main.go.
You may also run these:

      go build
     ./example

You should see (with a different path) the following output:

        $ go run main.go 
        /var/folders/kd/ygq5l_0d1xq1lzk_c7htft900000gn/T
        /tmp764135258/tmp588787953

If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

Creating temporary files and directories can be done using the ioutil package. Although you must still delete the files yourself, RemoveAll is the convention, and it will do that for you with only one extra line of code.

When writing tests, the use of temporary files is highly recommended. It's also useful for things such as build artifacts and more. The Go ioutil package will try and honor the OS preferences by default, but it allows you to fall back to other directories if required.

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1/ch01lvl1sec07/working-with-temporary-files","metaDescription":"","titleSlug":"working-with-temporary-files"},{"id":"ch01lvl1sec08","contentType":"text","title":"Working with text/template and HTML/templates","content":"

Working with text/template and HTML/templates

Go provides rich support for templates. It is simple to nest templates, import functions, represent variables, iterate over data, and so on. If you need something more sophisticated than a CSV writer, templates may be a great solution.

Another application for templates is for websites. When we want to render server-side data to the client, templates fit the bill nicely. At first, Go templates can appear confusing. This chapter will explore working with templates, collecting templates inside of a directory, and working with HTML templates.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

From your terminal/console application, create a new directory called chapter1/templates.
Navigate to this directory.
Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/templates, or use this as an exercise to write some of your own!
Create a file called templates.go with the following contents:

        package templates
        
        import (
                "os"
                "strings"
                "text/template"
        )
        
        const sampleTemplate = `
                This template demonstrates printing a {{ .Variable | 
                printf "%#v" }}.
         
                {{if .Condition}}
                If condition is set, we'll print this
                {{else}}
                Otherwise, we'll print this instead
                {{end}}
        
                Next we'll iterate over an array of strings:
                {{range $index, $item := .Items}}
                {{$index}}: {{$item}}
                {{end}}

                We can also easily import other functions like 
                strings.Split
                then immediately used the array created as a result:
                {{ range $index, $item := split .Words ","}}
                {{$index}}: {{$item}}
                {{end}}
        
                Blocks are a way to embed templates into one another
                {{ block "block_example" .}}
                No Block defined!
                {{end}}
        
        
                {{/*
                This is a way
                to insert a multi-line comment
                */}}
`
        
        const secondTemplate = `
                {{ define "block_example" }}
                {{.OtherVariable}}
                {{end}}
`        
        
        // RunTemplate initializes a template and demonstrates a 
        // variety of template helper functions
        func RunTemplate() error {
                data := struct {
                        Condition bool
                        Variable string
                        Items []string
                        Words string
                        OtherVariable string
                }{
                        Condition: true,
                        Variable: "variable",
                        Items: []string{"item1", "item2", "item3"},
                        Words: 
                        "another_item1,another_item2,another_item3",
                        OtherVariable: "I'm defined in a second 
                        template!",
                }
        
                funcmap := template.FuncMap{
                        "split": strings.Split,
                }
        
                // these can also be chained
                t := template.New("example")
                t = t.Funcs(funcmap)
        
                // We could use Must instead to panic on error
                // template.Must(t.Parse(sampleTemplate))
                t, err := t.Parse(sampleTemplate)
                if err != nil {
                        return err
                }
        
                // to demonstrate blocks we'll create another template
                // by cloning the first template, then parsing a second
                t2, err := t.Clone()
                if err != nil {
                        return err
                }
        
                t2, err = t2.Parse(secondTemplate)
                if err != nil {
                        return err
                }
        
                // write the template to stdout and populate it
                // with data
                err = t2.Execute(os.Stdout, &data)
                if err != nil {
                        return err
                }
        
                return nil
        }

Create a file called template_files.go with the following contents:

        package templates
        
        import (
                "io/ioutil"
                "os"
                "path/filepath"
                "text/template"
        )
        
        //CreateTemplate will create a template file that contains data
        func CreateTemplate(path string, data string) error {
                return ioutil.WriteFile(path, []byte(data), 
                os.FileMode(0755))
        }

        // InitTemplates sets up templates from a directory
        func InitTemplates() error {
                tempdir, err := ioutil.TempDir("", "temp")
                if err != nil {
                        return err
                }
                defer os.RemoveAll(tempdir)
        
                err = CreateTemplate(filepath.Join(tempdir, "t1.tmpl"), 
                `Template 1! {{ .Var1 }}
                {{ block "template2" .}} {{end}}
                {{ block "template3" .}} {{end}}
                `)
                if err != nil {
                        return err
                }
        
                err = CreateTemplate(filepath.Join(tempdir, "t2.tmpl"), 
                `{{ define "template2"}}Template 2! {{ .Var2 }}{{end}}
                `)
                if err != nil {
                        return err
                }

                err = CreateTemplate(filepath.Join(tempdir, "t3.tmpl"), 
                `{{ define "template3"}}Template 3! {{ .Var3 }}{{end}}
                `)
                if err != nil {
                        return err
                }
        
                pattern := filepath.Join(tempdir, "*.tmpl")

                // Parse glob will combine all the files that match 
                // glob and combine them into a single template
                tmpl, err := template.ParseGlob(pattern)
                if err != nil {
                        return err
                }
        
                // Execute can also work with a map instead
                // of a struct
                tmpl.Execute(os.Stdout, map[string]string{
                        "Var1": "Var1!!",
                        "Var2": "Var2!!",
                        "Var3": "Var3!!",
                 })

                 return nil
        }

Create a file called html_templates.go with the following content:

        package templates
        
        import (
                "fmt"
                "html/template"
                "os"
        )
        
        // HTMLDifferences highlights some of the differences
        // between html/template and text/template
        func HTMLDifferences() error {
                t := template.New("html")
                t, err := t.Parse("<h1>Hello! {{.Name}}</h1>n")
                if err != nil {
                        return err
         }
        
                // html/template auto-escapes unsafe operations like 
                // javascript injection this is contextually aware and 
                // will behave differently
                // depending on where a variable is rendered
                err = t.Execute(os.Stdout, map[string]string{"Name": "                 <script>alert('Can you see me?')</script>"})
                if err != nil {
                        return err
                }
        
                // you can also manually call the escapers
                fmt.Println(template.JSEscaper(`example         
                <example@example.com>`))
                fmt.Println(template.HTMLEscaper(`example 
                <example@example.com>`))
                fmt.Println(template.URLQueryEscaper(`example 
                <example@example.com>`))
        
                return nil
        }

Create a new directory named example.
Navigate to example.
Create a main.go file with the following contents and ensure that you modify the tempfiles imported to use the path you set up in step 2:

        package main
        
        import "github.com/agtorre/go-cookbook/chapter1/templates"
        
        func main() {
                if err := templates.RunTemplate(); err != nil {
                        panic(err)
                }
        
                if err := templates.InitTemplates(); err != nil {
                        panic(err)
                }
        
                if err := templates.HTMLDifferences(); err != nil {
                        panic(err)
                }
        }

Run go run main.go.
You may also run these:

      go build
      ./example

You should see (with a different path) the following output:

        $ go run main.go 

         This template demonstrates printing a "variable".


         If condition is set, we'll print this


         Next we'll iterate over an array of strings:

         0: item1

         1: item2

         2: item3


         We can also easily import other functions like strings.Split
         then immediately used the array created as a result:

         0: another_item1

         1: another_item2

         2: another_item3


         Blocks are a way to embed templates into one another

         I'm defined in a second template!





         Template 1! Var1!!
         Template 2! Var2!!
         Template 3! Var3!!
         <h1>Hello! &lt;script&gt;alert('Can you see 
         me?')&lt;/script&gt;</h1>
        example x3Cexample@example.comx3E
        example &lt;example@example.com&gt;
        example+%3Cexample%40example.com%3E

If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

Go has two template packages--text/template and html/template. These share functionality and a variety of functions. In general, use html/template to render websites and text/html for everything else. Templates are plain text, but variables and functions can be used inside of curly brace blocks.

The template packages also provide convenience methods to work with files. The example creates a number of templates in a temporary directory and then reads them all with a single line of code.

The html/template package is a wrapper around the text/template package. All of the template examples work with the html/template package directly, using no modification and only changing the import statement. HTML templates provide the added benefit of context-aware safety. This prevents things such as JavaScript injection.

The template packages provide what you'd expect out of a modern template library. It's easy to combine templates, add application logic, and ensure safety when emitting results to HTML and JavaScript.

","url":"https://subscription.packtpub.com/book/programming/9781783286836/1/ch01lvl1sec08/working-with-text-template-and-html-templates","metaDescription":"","titleSlug":"working-with-text-template-and-html-templates"}]

Go Cookbook

By : Aaron Torres

Go Cookbook

By: Aaron Torres

Overview of this book

Related Content you might be interested in

Current Title:

Go Cookbook

Go Standard Library Cookbook

Hands-On System Programming with Go

Building RESTful Web services with Go