flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From burakkk <burak.isi...@gmail.com>
Subject Transferring another server using flume
Date Wed, 29 Jan 2014 14:24:25 GMT
Hi folks,
I have question about flume-ng. There are some different generating log
machines. These log files are small (around 4-5mb per file). I want to get
or read these files into my main server from these remote servers on
a specific directory and then I want to put it into HDFS. I can't install
any kind of application on these remote servers so that I can't use avro
and thrift source.

For now I use rsync to sync files between two different machines and put
them using hdfs file commands such as hdfs fs -put. But there are some
issues about rsync.

In order to solve this problem, what kind of source should I use and how
can I do that?

Best Regards...


*BURAK ISIKLI* | *http://burakisikli.wordpress.com

View raw message