hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Which LZO library to use?
Date Tue, 03 Aug 2010 00:02:09 GMT
On Mon, Aug 2, 2010 at 11:12 AM, Alex Kozlov <alexvk@cloudera.com> wrote:

> The code is currently maintained by Kevin Weil and Todd Lipcon.  For
> completeness, there is one more distribution at
> http://github.com/toddlipcon/hadoop-lzo.  AFAIK, the Todd Lipcon's and
> Kevin
> Weil's distribution are synced.
>

Yep, Kevin and I pull from each other whenever we make any changes. So
sometimes a bug fix will hit one of our repos a day or two before the other,
but we keep in touch and should never diverge.



>
> Most of the differences with google's code are bug fixes: the lzo file
> format itself had not changed and you can actually read the files created
> with lzop (the LZO command line tool).   Their are no version compatibility
> issues currently.
>
> Alex K
>
> On Mon, Aug 2, 2010 at 3:25 AM, Eran Kutner <eran@gigya.com> wrote:
>
> > Hi,
> > I want to enable LZO compression on my cluster but see there are a few
> > alternatives and the wiki page itself is very confusing so it's not clear
> > what is the right choice. I was looking at this page:
> > http://wiki.apache.org/hadoop/UsingLzoCompression, at the top it
> > recommends
> > using Kevin Weil's version (which seems to be the same one released by
> > Twitter) but warns it doesn't contain all fixes and lower in the article
> it
> > refers to the original google code repository (
> > http://code.google.com/p/hadoop-gpl-compression/).
> > The thing the concerns me most is future compatibility, so whichever
> > library
> > I pick now I want to be certain my data compressed will still be readable
> > when I I upgrade to the next major version of Hadoop and Hbase. It seems
> > that only the Google code project has newer releases compatible with
> future
> > version of Hadoop.
> >
> > So I'm looking for recommendations on which library to use.
> >
> >
> > Thanks,
> > Eran
> >
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message