hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bobby Dennett <bdennett+softw...@gmail.com>
Subject Enabling LZO compression of map outputs in Cloudera Hadoop 0.20.1
Date Thu, 05 Aug 2010 22:52:19 GMT
We are looking to enable LZO compression of the map outputs on our
Cloudera 0.20.1 cluster. It seems there are various sets of
instructions available and I am curious what your thoughts are
regarding which one would be best for our Hadoop distribution and OS
(Ubuntu 8.04 64-bit). In particular, hadoop-gpl-compression
(http://code.google.com/p/hadoop-gpl-compression) vs. hadoop-lzo
(http://github.com/kevinweil/hadoop-lzo).

Some of what appear to be the better instructions/guides out there:
* Josh Patterson's reply on June 25th to the "Newbie to HDFS
compression" thread --
http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201006.mbox/%3CAANLkTileo-q8USEiP8Y3Na9pDYHlyUFIPpR0In0LkpJm@mail.gmail.com%3E
* hadoop-gpl-compression FAQ --
http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ
* "Hadoop at Twitter (part 1): Splittable LZO Compression" blog post
-- http://www.cloudera.com/blog/2009/11/hadoop-at-twitter-part-1-splittable-lzo-compression/

Thanks in advance,
-Bobby

Mime
View raw message