hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mirko.kaempf" <mirko.kae...@gmail.com>
Subject AW: Best way to transfer data to HDFS
Date Fri, 06 Jun 2014 14:48:24 GMT
Hi,
Flume is not limitted to long data.  It has a flume-source to observe directories and can
load existing files. The morphlines from kite-sdk can help you even transform data on the
fly. 
Cheers,
Mirko



Von Samsung Mobile gesendet

<div>-------- Ursprüngliche Nachricht --------</div><div>Von: Shashidhar
Rao <raoshashidhar123@gmail.com> </div><div>Datum:06.06.2014  16:26  (GMT+01:00)
</div><div>An: user@hadoop.apache.org </div><div>Betreff: Best way
to transfer data to HDFS </div><div>
</div>Hi Hadoop Experts,

I have to transfer 5 yrs of historical billing data to the tune of 25 -30 TB to HDFS to be
later analyzed by Map Reduce program. Sqoop is out of question as these files are not residing
in OLTP and so is Flume as these are not log files generated by App Server.

What tools are available to transfer such Big data? Would it be prudent to use Hadoop put
command to transfer these files

Regards
Shashi
Mime
View raw message