Book Image

Hands-On Software Engineering with Golang

By : Achilleas Anagnostopoulos
Book Image

Hands-On Software Engineering with Golang

By: Achilleas Anagnostopoulos

Overview of this book

Over the last few years, Go has become one of the favorite languages for building scalable and distributed systems. Its opinionated design and built-in concurrency features make it easy for engineers to author code that efficiently utilizes all available CPU cores. This Golang book distills industry best practices for writing lean Go code that is easy to test and maintain, and helps you to explore its practical implementation by creating a multi-tier application called Links ‘R’ Us from scratch. You’ll be guided through all the steps involved in designing, implementing, testing, deploying, and scaling an application. Starting with a monolithic architecture, you’ll iteratively transform the project into a service-oriented architecture (SOA) that supports the efficient out-of-core processing of large link graphs. You’ll learn about various cutting-edge and advanced software engineering techniques such as building extensible data processing pipelines, designing APIs using gRPC, and running distributed graph processing algorithms at scale. Finally, you’ll learn how to compile and package your Go services using Docker and automate their deployment to a Kubernetes cluster. By the end of this book, you’ll know how to think like a professional software developer or engineer and write lean and efficient Go code.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Software Engineering with Golang
Achilleas Anagnostopoulos
Jan, 2020 | 640 pages
Small book image
Effective Concurrency in Go
Burak Serdar
Apr, 2023 | 212 pages
Table of Contents (21 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
1
Section 1: Software Engineering and the Software Development Life Cycle
Section 1: Software Engineering and the Software Development Life Cycle
Free Chapter
2
A Bird's-Eye View of Software Engineering
A Bird's-Eye View of Software Engineering
What is software engineering?
Types of software engineering roles
A list of software development models that all engineers should know
Summary
Questions
Further reading
3
Section 2: Best Practices for Maintainable and Testable Go Code
Section 2: Best Practices for Maintainable and Testable Go Code
4
Best Practices for Writing Clean and Maintainable Go Code
Best Practices for Writing Clean and Maintainable Go Code
The SOLID principles of object-oriented design
Organizing code into packages
Tips and tools for writing lean and easy-to-maintain Go code
Summary
Questions
Further reading
5
Dependency Management
Dependency Management
What's all the fuss about software versioning?
Vendoring – the good, the bad, and the ugly
Strategies and tools for vendoring dependencies
Summary
Questions
Further reading
6
The Art of Testing
The Art of Testing
Technical requirements
Unit testing
Integration versus functional testing
Smoke tests
Chaos testing – breaking your systems in fun and interesting ways!
Tips and tricks for writing tests
Summary
Questions
Further reading
7
Section 3: Designing and Building a Multi-Tier System from Scratch
Section 3: Designing and Building a Multi-Tier System from Scratch
8
The Links 'R'; Us Project
The Links 'R'; Us Project
System overview – what are we going to be building?
Selecting an SDLC model for our project
Requirements analysis
System component modeling
Monolith or microservices? The ultimate question
Summary
Questions
Further reading
9
Building a Persistence Layer
Building a Persistence Layer
Technical requirements
Exploring a taxonomy of database systems
Understanding the need for a data layer abstraction
Designing the data layer for the link graph component
Designing the data layer for the text indexer component
Summary
Questions
Further reading
10
Data-Processing Pipelines
Data-Processing Pipelines
Technical requirements
Building a generic data-processing pipeline in Go
Building a crawler pipeline for the Links 'R' Us project
Summary
Questions
Further reading
11
Graph-Based Data Processing
Graph-Based Data Processing
Technical requirements
Exploring the Bulk Synchronous Parallel model
Building a graph processing system in Go
Solving interesting graph problems
Summary
Further reading
12
Communicating with the Outside World
Communicating with the Outside World
Technical requirements
Designing robust, secure, and backward-compatible REST APIs
Building RPC-based APIs with the help of gRPC
Decoupling Links 'R' Us components from the underlying data stores
Summary
Questions
Further reading
13
Building, Packaging, and Deploying Software
Building, Packaging, and Deploying Software
Technical requirements
Building and packaging Go services using Docker
A gentle introduction to Kubernetes
Building and deploying a monolithic version of Links 'R' Us
Summary
Questions
Further reading
14
Section 4: Scaling Out to Handle a Growing Number of Users
Section 4: Scaling Out to Handle a Growing Number of Users
15
Splitting Monoliths into Microservices
Splitting Monoliths into Microservices
Technical requirements
Monoliths versus service-oriented architectures
Monitoring the state of your microservices
Building a microservice-based version of Links 'R' Us
Summary
Questions
Further reading
16
Building Distributed Graph-Processing Systems
Building Distributed Graph-Processing Systems
Technical requirements
Introducing the master/worker model
Out-of-core distributed graph processing
Deploying a distributed version of the Links 'R' Us PageRank calculator
Summary
Questions
Further reading
17
Metrics Collection and Visualization
Metrics Collection and Visualization
Technical requirements
Monitoring from the perspective of a site reliability engineer
Exploring options for collecting and aggregating metrics
Visualizing collected metrics using Grafana
Using Prometheus as an end-to-end solution for alerting
Summary
Questions
Further reading
18
Epilogue
Epilogue
19
Assessments
Assessments
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
20
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Out-of-core distributed graph processing

Back in Chapter 8, Graph-Based Data Processing, we designed and built our very own system for implementing graph-based algorithms based on the Bulk Synchronous Parallel (BSP) model. Admittedly, our final implementation was heavily influenced by the ideas from the Google paper describing Pregel [4], a system that was originally built by Google engineers to tackle graph-based computation at scale.

While the bspgraph package from Chapter 8, Graph-Based Data Processing, can automatically distribute the graph computation load among a pool of workers, it is still limited to running on a single compute node. As our Links 'R' Us crawler augments our link index with more and more links, we will eventually reach a point where the PageRank computation will simply take too long. Updating the PageRank scores for the entire graphs might take...

Previous Section
End of Section 4
Next Section