hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kozlov <ale...@cloudera.com>
Subject Re: Which LZO library to use?
Date Mon, 02 Aug 2010 18:12:06 GMT
The code is currently maintained by Kevin Weil and Todd Lipcon.  For
completeness, there is one more distribution at
http://github.com/toddlipcon/hadoop-lzo.  AFAIK, the Todd Lipcon's and Kevin
Weil's distribution are synced.

Most of the differences with google's code are bug fixes: the lzo file
format itself had not changed and you can actually read the files created
with lzop (the LZO command line tool).   Their are no version compatibility
issues currently.

Alex K

On Mon, Aug 2, 2010 at 3:25 AM, Eran Kutner <eran@gigya.com> wrote:

> Hi,
> I want to enable LZO compression on my cluster but see there are a few
> alternatives and the wiki page itself is very confusing so it's not clear
> what is the right choice. I was looking at this page:
> http://wiki.apache.org/hadoop/UsingLzoCompression, at the top it
> recommends
> using Kevin Weil's version (which seems to be the same one released by
> Twitter) but warns it doesn't contain all fixes and lower in the article it
> refers to the original google code repository (
> http://code.google.com/p/hadoop-gpl-compression/).
> The thing the concerns me most is future compatibility, so whichever
> library
> I pick now I want to be certain my data compressed will still be readable
> when I I upgrade to the next major version of Hadoop and Hbase. It seems
> that only the Google code project has newer releases compatible with future
> version of Hadoop.
>
> So I'm looking for recommendations on which library to use.
>
>
> Thanks,
> Eran
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message