Time for action – writing to multiple sinks
We need one final piece of capability to build such networks, namely, an agent that can write to multiple sinks. Let's create one.
Create the following configuration file as
agent7.conf
:agent7.sources = netsource agent7.sinks = hdfssink filesink agent7.channels = memorychannel1 memorychannel2 agent7.sources.netsource.type = netcat agent7.sources.netsource.bind = localhost agent7.sources.netsource.port = 3000 agent7.sources.netsource.interceptors = ts agent7.sources.netsource.interceptors.ts.type = org.apache.flume.interceptor.TimestampInterceptor$Builder agent7.sinks.hdfssink.type = hdfs agent7.sinks.hdfssink.hdfs.path = /flume-%Y-%m-%d agent7.sinks.hdfssink.hdfs.filePrefix = log agent7.sinks.hdfssink.hdfs.rollInterval = 0 agent7.sinks.hdfssink.hdfs.rollCount = 3 agent7.sinks.hdfssink.hdfs.fileType = DataStream agent7.sinks.filesink.type = FILE_ROLL agent7.sinks.filesink.sink.directory = /home/hadoop/flume/files agent7.sinks.filesink.sink.rollInterval...