We need one final piece of capability to build such networks, namely, an agent that can write to multiple sinks. Let's create one.
Create the following configuration file as
agent7.conf
:agent7.sources = netsource agent7.sinks = hdfssink filesink agent7.channels = memorychannel1 memorychannel2 agent7.sources.netsource.type = netcat agent7.sources.netsource.bind = localhost agent7.sources.netsource.port = 3000 agent7.sources.netsource.interceptors = ts agent7.sources.netsource.interceptors.ts.type = org.apache.flume.interceptor.TimestampInterceptor$Builder agent7.sinks.hdfssink.type = hdfs agent7.sinks.hdfssink.hdfs.path = /flume-%Y-%m-%d agent7.sinks.hdfssink.hdfs.filePrefix = log agent7.sinks.hdfssink.hdfs.rollInterval = 0 agent7.sinks.hdfssink.hdfs.rollCount = 3 agent7.sinks.hdfssink.hdfs.fileType = DataStream agent7.sinks.filesink.type = FILE_ROLL agent7.sinks.filesink.sink.directory = /home/hadoop/flume/files agent7.sinks.filesink.sink.rollInterval...