Book Image

Apache Flume: Distributed Log Collection for Hadoop

By : Steven Hoffman
Book Image

Apache Flume: Distributed Log Collection for Hadoop

By: Steven Hoffman

Overview of this book

Table of Contents (16 chapters)
Apache Flume: Distributed Log Collection for Hadoop Second Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Channel selectors


As we discussed in Chapter 1, Overview and Architecture, a source can write to one or more channels. This is why the property is plural (channels instead of channel). There are two ways multiple channels can be handled. The event can be written to all the channels or to just one channel, based on some Flume header value. The internal mechanism for this in Flume is called a channel selector.

The selector for any channel can be specified using the selector.type property. All selector-specific properties begin with the usual Source prefix: the agent name, keyword sources, and source name:

agent.sources.s1.selector.type=replicating

Replicating

If you do not specify a selector for a source, replicating is the default. The replicating selector writes the same event to all channels in the source's channels list:

agent.sources.s1.channels=c1 c2 c3
agent.sources.s1.selector.type=replicating

In this example, every event will be written to all three channels: c1, c2, and c3.

There is an...