Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Apache Flume: Distributed Log Collection for Hadoop - Second Edition
  • Table Of Contents Toc
  • Feedback & Rating feedback
Apache Flume: Distributed Log Collection for Hadoop - Second Edition

Apache Flume: Distributed Log Collection for Hadoop - Second Edition - Second Edition

By : Steven Hoffman
5 (1)
close
close
Apache Flume: Distributed Log Collection for Hadoop - Second Edition

Apache Flume: Distributed Log Collection for Hadoop - Second Edition

5 (1)
By: Steven Hoffman

Overview of this book

If you are a Hadoop programmer who wants to learn about Flume to be able to move datasets into Hadoop in a timely and replicable manner, then this book is ideal for you. No prior knowledge about Apache Flume is necessary, but a basic knowledge of Hadoop and the Hadoop File System (HDFS) is assumed.
Table of Contents (11 chapters)
close
close
10
Index

Monitoring performance metrics


Now that we have covered some options for process monitoring, how do you know whether your application is actually doing the work you think it is? On many occasions, I've seen a stuck syslog-ng process that appears to be running, but it just wasn't sending any data. I'm not picking on syslog-ng specifically; all software does this when conditions that are not designed for occur.

When talking about Flume data flows, you need to monitor the following:

  • Data entering sources is within expected rates

  • Data isn't overflowing your channels

  • Data is exiting sinks at expected rates

Flume has a pluggable monitoring framework, but as mentioned at the beginning of the chapter, it is still very much a work in progress. This does not mean you shouldn't use it, as that would be foolish. It means you'll want to prepare extra testing and integration time anytime you upgrade.

Note

While not covered in the Flume documentation, it is common to enable JMX in your Flume JVM (http://bit.ly...

Visually different images
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Apache Flume: Distributed Log Collection for Hadoop - Second Edition
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon