Time for action – multi level Flume networks

Let's put together a few pieces we touched on earlier and see how one Flume agent can use another as its sink.

1. Create the following file as agent6.conf:

   agent6.sources = avrosource
   agent6.sinks = avrosink
   agent6.channels = memorychannel
   
   agent6.sources.avrosource.type = avro
   agent6.sources.avrosource.bind = localhost
   agent6.sources.avrosource.port = 2000
   agent6.sources.avrosource.threads = 5
   
   agent6.sinks.avrosink.type = avro
   agent6.sinks.avrosink.hostname = localhost
   agent6.sinks.avrosink.port = 4000
   
   agent6.channels.memorychannel.type = memory
   agent6.channels.memorychannel.capacity = 1000
   agent6.channels.memorychannel.transactionCapacity = 100
   
   agent6.sources.avrosource.channels = memorychannel
   agent6.sinks.avrosink.channel = memorychannel

2. Start an agent configured as per the agent3.conf file created earlier, that is, with an Avro source and a file sink:

   $ flume-ng client –conf conf –conf-file agent3.conf agent3  
   

3. In a second window, start another...