hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <ramon....@accenture.com>
Subject RE: What's the best way to compress a folder in hadoop?
Date Fri, 29 Jun 2012 12:38:26 GMT
Félix, I think your are looking for hadoop har files. http://hadoop.apache.org/mapreduce/docs/r0.21.0/hadoop_archives.html

From: Félix López [mailto:jaaaelpumuki@gmail.com]
Sent: viernes, 29 de junio de 2012 13:31
To: hdfs-user@hadoop.apache.org
Subject: Re: What's the best way to compress a folder in hadoop?

thanks, I've read but that is about compressing a files, I'm talking about compressing folders
with subfolders and files.
In fact i have a mapreduce task for compressing a folder:
2012/6/29 madhu phatak <phatak.dev@gmail.com<mailto:phatak.dev@gmail.com>>
 Please refer to the discussion here  http://stackoverflow.com/questions/7153087/hadoop-compress-file-in-hdfs

On Fri, Jun 29, 2012 at 1:07 PM, Félix López <jaaaelpumuki@gmail.com<mailto:jaaaelpumuki@gmail.com>>
The folder contains files with text and other folders with text files. The text is not key/value,
it's just text. Something like this:
Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has
been the industry's standard dumm...

I'm thinking about creating a new Hdfs command to zip in hadoop, but i'm not sure whether
hadoop distributes the execution, because otherwise it may takes a long time and very cpu

Any ideas?




It's easier to ask forgiveness than it is to get permission

".....it doesn't matter how many times you fail. It doesn't matter how many times you almost
get it right. No one is going to know or care about your failures, and either should you.
All you have to do is learn from them and those around you because...All that matters in business
is that you get it right once. Then everyone can tell you how lucky you are."
--Mark Cuban"

"Always be the worst guy in every band you're in." If you're the best guy there, you need
to be in a different band. And I think that works for almost everything that's out there as
well." Pat Metheny

Subject to local law, communications with Accenture and its affiliates including telephone
calls and emails (including content), may be monitored by our systems for the purposes of
security and the assessment of internal compliance with Accenture policy.


View raw message