Time for action – capturing the output of a command to a flat file

Let's show this in action, along the way demonstrating a new kind of source as well.

1. Create the following file as agent2.conf within the Flume working directory:

   agent2.sources = execsource
   agent2.sinks = filesink
   agent2.channels = filechannel
   
   agent2.sources.execsource.type = exec
   agent2.sources.execsource.command = cat /home/hadoop/message
   
   agent2.sinks.filesink.type = FILE_ROLL
   agent2.sinks.filesink.sink.directory = /home/hadoop/flume/files
   agent2.sinks.filesink.sink.rollInterval = 0
   
   agent2.channels.filechannel.type = file
   agent2.channels.filechannel.checkpointDir = /home/hadoop/flume/fc/checkpoint
   agent2.channels.filechannel.dataDirs = /home/hadoop/flume/fc/data
   
   agent2.sources.execsource.channels = filechannel
   agent2.sinks.filesink.channel = filechannel

2. Create a simple test file in the home directory:

   $ echo "Hello again Flume!" > /home/hadoop/message
   

3. Start the agent:

   $ flume-ng agent --conf conf --conf-file agent2...