Time for action – capturing a remote file in a local flat file

Let's show another example of capturing data to a file sink. This time we will use another Flume capability that allows it to receive data from a remote client.

1. Create the following file as agent3.conf in the Flume working directory:

   agent3.sources = avrosource
   agent3.sinks = filesink
   agent3.channels = jdbcchannel
   
   agent3.sources.avrosource.type = avro
   agent3.sources.avrosource.bind = localhost
   agent3.sources.avrosource.port = 4000
   agent3.sources.avrosource.threads = 5
   
   agent3.sinks.filesink.type = FILE_ROLL
   agent3.sinks.filesink.sink.directory = /home/hadoop/flume/files
   agent3.sinks.filesink.sink.rollInterval = 0
   
   agent3.channels.jdbcchannel.type = jdbc
   
   agent3.sources.avrosource.channels = jdbcchannel
   agent3.sinks.filesink.channel = jdbcchannel

2. Create a new test file as /home/hadoop/message2:

   Hello from Avro!

3. Start the Flume agent:

   $ flume-ng agent –conf conf –conf-file agent3.conf –name agent3 
   

4. In another window, use the Flume...