Book Image

Hands-On Infrastructure Monitoring with Prometheus

By : Joel Bastos, Pedro Araújo
Book Image

Hands-On Infrastructure Monitoring with Prometheus

By: Joel Bastos, Pedro Araújo

Overview of this book

Prometheus is an open source monitoring system. It provides a modern time series database, a robust query language, several metric visualization possibilities, and a reliable alerting solution for traditional and cloud-native infrastructure. This book covers the fundamental concepts of monitoring and explores Prometheus architecture, its data model, and how metric aggregation works. Multiple test environments are included to help explore different configuration scenarios, such as the use of various exporters and integrations. You’ll delve into PromQL, supported by several examples, and then apply that knowledge to alerting and recording rules, as well as how to test them. After that, alert routing with Alertmanager and creating visualizations with Grafana is thoroughly covered. In addition, this book covers several service discovery mechanisms and even provides an example of how to create your own. Finally, you’ll learn about Prometheus federation, cross-sharding aggregation, and also long-term storage with the help of Thanos. By the end of this book, you’ll be able to implement and scale Prometheus as a full monitoring system on-premises, in cloud environments, in standalone instances, or using container orchestration with Kubernetes.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Infrastructure Monitoring with Prometheus
Joel Bastos | Pedro Araújo
May, 2019 | 442 pages
Small book image
Observability with Grafana
Peter Holmes | Rob Chapman
Jan, 2024 | 356 pages
Small book image
The DevOps 2.2 Toolkit
Viktor Farcic
Mar, 2018 | 360 pages
Small book image
The DevOps 2.5 Toolkit
Viktor Farcic
Nov, 2019 | 322 pages
Table of Contents (21 chapters)
Preface
Preface
Introduction to the book and the technology
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Introduction
Section 1: Introduction
2
Monitoring Fundamentals
Monitoring Fundamentals
Definition of monitoring
Whitebox versus blackbox monitoring
Understanding metrics collection
Summary
Questions
Further reading
3
An Overview of the Prometheus Ecosystem
An Overview of the Prometheus Ecosystem
Metrics collection with Prometheus
Exposing internal state with exporters
Alert routing and management with Alertmanager
Visualizing your data
Summary
Questions
Further reading
4
Setting Up a Test Environment
Setting Up a Test Environment
Code organization
Machine requirements
Spinning up a new environment
Summary
Questions
Further reading
5
Section 2: Getting Started with Prometheus
Section 2: Getting Started with Prometheus
6
Prometheus Metrics Fundamentals
Prometheus Metrics Fundamentals
Understanding the Prometheus data model
A tour of the four core metric types
Longitudinal and cross-sectional aggregations
Summary
Questions
Further reading
7
Running a Prometheus Server
Running a Prometheus Server
Deep dive into the Prometheus configuration
Managing Prometheus in a standalone server
Managing Prometheus in Kubernetes
Summary
Questions
Further reading
8
Exporters and Integrations
Exporters and Integrations
Test environments for this chapter
Operating system exporter
Container exporter
From logs to metrics
Blackbox monitoring
Pushing metrics
More exporters
Summary
Questions
Further reading
9
Prometheus Query Language - PromQL
Prometheus Query Language - PromQL
The test environment for this chapter
Getting to know the basics of PromQL
Common patterns and pitfalls
Moving on to more complex queries
Summary
Questions
Further reading
10
Troubleshooting and Validation
Troubleshooting and Validation
The test environment for this chapter
Exploring promtool
Logs and endpoint validation
Analyzing the time series database
Summary
Questions
Further reading
11
Section 3: Dashboards and Alerts
Section 3: Dashboards and Alerts
12
Defining Alerting and Recording Rules
Defining Alerting and Recording Rules
Creating the test environment
Understanding how rule evaluation works
Setting up alerting in Prometheus
Testing your rules
Summary
Questions
Further reading
13
Discovering and Creating Grafana Dashboards
Discovering and Creating Grafana Dashboards
Test environment for this chapter
How to use Grafana with Prometheus
Building your own dashboards
Discovering ready-made dashboards
Default Prometheus visualizations
Summary
Questions
Further reading
14
Understanding and Extending Alertmanager
Understanding and Extending Alertmanager
Setting up the test environment
Alertmanager fundamentals
Alertmanager configuration
Common Alertmanager notification integrations
Customizing your alert notifications
Who watches the Watchmen?
Summary
Questions
Further reading
15
Section 4: Scalability, Resilience, and Maintainability
Section 4: Scalability, Resilience, and Maintainability
16
Choosing the Right Service Discovery
Choosing the Right Service Discovery
Test environment for this chapter
Running through the service discovery options
Using a built-in service discovery
Building a custom service discovery
Summary
Questions
Further reading
17
Scaling and Federating Prometheus
Scaling and Federating Prometheus
Test environment for this chapter
Scaling with the help of sharding
Having a global view using federation
Using Thanos to mitigate Prometheus shortcomings at scale
Summary
Questions
Further reading
18
Integrating Long-Term Storage with Prometheus
Integrating Long-Term Storage with Prometheus
Test environment for this chapter
Remote write and remote read
Options for metrics storage
Thanos remote storage and ecosystem
Summary
Questions
Further reading
19
Assessments
Assessments
Chapter 1, Monitoring Fundamentals
Chapter 2, An Overview of the Prometheus Ecosystem
Chapter 3, Setting Up a Test Environment
Chapter 4, Prometheus Metrics Fundamentals
Chapter 5, Running a Prometheus Server
Chapter 6, Exporters and Integrations
Chapter 7, Prometheus Query Language - PromQL
Chapter 8, Troubleshooting and Validation
Chapter 9, Defining Alerting and Recording Rules
Chapter 10, Discovering and Creating Grafana Dashboards
Chapter 11, Understanding and Extending Alertmanager
Chapter 12, Choosing the Right Service Discovery
Chapter 13, Scaling and Federating Prometheus
Chapter 14, Integrating Long-Term Storage with Prometheus
20
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Visualizing your data

Data visualization is one of the simplest ways to produce or consume information. Prometheus exposes a well-defined API, where PromQL queries can produce raw data for visualizations.

Currently, the best external software for visualization is Grafana, which we will explain thoroughly in Chapter 10, Discovering and Creating Grafana Dashboards. The Grafana team has made its integration with Prometheus seamless, and the result is a delightful user experience.

The Prometheus server also ships with two internal visualizations components:

  • Expression browser: Here, you can run PromQL directly to quickly query data and visualize it instantly:
Figure 2.4: The Prometheus expression browser interface
  • Consoles: These are web pages that are built using the Golang templating language and are served by the Prometheus server itself. This approach allows you to have...
Previous Section
End of Section 5
Next Section