hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bobby Dennett <bdennett+softw...@gmail.com>
Subject Enabling LZO compression of map outputs in Cloudera Hadoop 0.20.1
Date Thu, 05 Aug 2010 22:52:19 GMT
We are looking to enable LZO compression of the map outputs on our
Cloudera 0.20.1 cluster. It seems there are various sets of
instructions available and I am curious what your thoughts are
regarding which one would be best for our Hadoop distribution and OS
(Ubuntu 8.04 64-bit). In particular, hadoop-gpl-compression
(http://code.google.com/p/hadoop-gpl-compression) vs. hadoop-lzo

Some of what appear to be the better instructions/guides out there:
* Josh Patterson's reply on June 25th to the "Newbie to HDFS
compression" thread --
* hadoop-gpl-compression FAQ --
* "Hadoop at Twitter (part 1): Splittable LZO Compression" blog post
-- http://www.cloudera.com/blog/2009/11/hadoop-at-twitter-part-1-splittable-lzo-compression/

Thanks in advance,

View raw message