hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From felix gao <gre1...@gmail.com>
Subject Question about copyFromLocal
Date Tue, 14 Dec 2010 21:14:44 GMT
Hi all,

I have couple of boxes that need to periodically copy stuff from their local
boxes to HDFS using HDFS Client by issuing hadoop fs -copyFromLocal src dest
command on it.  The file size is rather large and I am wondering is there
anyway to make hadoop transmit the data compressed and when it arrives at
the master, automatically compress it?  Something like the intermediate
compression from mapper to reducer but rather from a hdfs client to master.



View raw message