flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Something Something <mailinglist...@gmail.com>
Subject Import files from a directory on remote machine
Date Thu, 17 Apr 2014 00:14:18 GMT

Needless to say I am newbie to Flume, but I've got a basic flow working in
which I am importing a log file from my linux box to hdfs.  I am using

a1.sources.r1.command = tail -F /var/log/xyz.log

which is working like a stream of messages.  This is good!

Now what I want to do is copy log files from a directory on a remote
machine on a regular basis.  For example:

username@machinename:/var/log/logdir/<multiple files>

One way to do it is to simply 'scp' files from the remote directory into my
box on a regular basis, but what's the best way to do this in Flume?
Please let me know.

Thanks for the help.

View raw message