Book Image

Datadog Cloud Monitoring Quick Start Guide

By : Thomas Kurian Theakanath
Book Image

Datadog Cloud Monitoring Quick Start Guide

By: Thomas Kurian Theakanath

Overview of this book

Datadog is an essential cloud monitoring and operational analytics tool which enables the monitoring of servers, virtual machines, containers, databases, third-party tools, and application services. IT and DevOps teams can easily leverage Datadog to monitor infrastructure and cloud services, and this book will show you how. The book starts by describing basic monitoring concepts and types of monitoring that are rolled out in a large-scale IT production engineering environment. Moving on, the book covers how standard monitoring features are implemented on the Datadog platform and how they can be rolled out in a real-world production environment. As you advance, you'll discover how Datadog is integrated with popular software components that are used to build cloud platforms. The book also provides details on how to use monitoring standards such as Java Management Extensions (JMX) and StatsD to extend the Datadog platform. Finally, you'll get to grips with monitoring fundamentals, learn how monitoring can be rolled out using Datadog proactively, and find out how to extend and customize the Datadog platform. By the end of this Datadog book, you will have gained the skills needed to monitor your cloud infrastructure and the software applications running on it using Datadog.
Table of Contents (19 chapters)
Section 1: Getting Started with Datadog
Section 2: Extending Datadog
Section 3: Advanced Monitoring

Processing logs

Once the logs are collected by Datadog, they will be available on Log Explorer to view and search. The logs in the structured JSON format are processed by Datadog automatically. The unstructured logs can be processed further and analytical insights extracted. Datadog uses Pipelines and Processors to process the incoming logs.

To view the pipelines available, open the Logs | Configuration menu option from the main menu. The pipelines are listed under the Pipelines tab, as shown in the following sample screenshot:

Figure 13.2 – Pipelines for log processing

A pipeline can be fed with a subset of logs and it uses a set of processors that are executed sequentially for processing the logs. There are predefined pipelines available based on the Datadog integrations currently in use, and they are enabled if related logs are collected by Datadog. It is also possible to set up custom pipelines to meet specific indexing requirements.

In the...