hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vincent cai <clb991...@gmail.com>
Subject question about zip and extract jobs on hadoop
Date Thu, 16 Sep 2010 07:51:24 GMT
Hi all

   I'm just thinking about the elt extract jobs.

   is it possible to deploy that on hadoop cluster?

   if zip or unzip command can be run on hadoop datanodes , the network
bandwidth will be the only bottleneck.

   Millions of zip files distributed to datanodes and zip or unzip. if
possible make the super datanode by VM and SAN.

   That could be a "Super fast SAN"

   looks like the FileUtil class is containing some methods calling the
linux gzip or untar command, but the hadoop fs manual is not providing that

   pls let me know if you have any comments .

   I'm just thinking about the possibility.


Best Regards
Skype , cailibing1

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message