hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Sreekumar <hsreeku...@clickable.com>
Subject Re: LZO Compression changes in 0.90 ?
Date Fri, 01 Apr 2011 12:52:26 GMT
Hi,

I had faced the same error. I rebuilt hadoop-gpl-compression individually on
all nodes, copied the files to $HBASE_HOME/lib/native as well apart from
$HADOOP_HOME/lib/native and it started working. Is that fine? Or should I
recompile from github as suggested above?

Thanks,
Hari

On Tue, Mar 29, 2011 at 1:20 PM, Todd Lipcon <todd@cloudera.com> wrote:

> Yep. Kevin has apparently gotten too many job promotions recently, so it's
> mostly me maintaining it these days :) But he usually reviews and pulls my
> changes within a few days.
>
> -Todd
>
> On Mon, Mar 28, 2011 at 9:05 AM, Stack <stack@duboce.net> wrote:
>
> > It doesn't matter IIUC (Correct me if I'm wrong Todd).  They keep up
> > on each others git repo changes.
> > St.Ack
> >
> > On Mon, Mar 28, 2011 at 8:57 AM, George P. Stathis <gstathis@traackr.com
> >
> > wrote:
> > > BTW, which github version should I be cloning? Todd's or Kevin's?
> > >
> > > On Mon, Mar 28, 2011 at 11:35 AM, Friso van Vollenhoven
> > > <fvanvollenhoven@xebia.com> wrote:
> > >>
> > >> Yes, things changed a bit. The compressors now have a reinit(...)
> method
> > >> that allows HBase to reuse the objects. I believe Hadoop still works
> > >>
> > >> Here's the version that should be good:
> > >> https://github.com/toddlipcon/hadoop-lzo
> > >>
> > >> If you're already on a recent version, then something else might have
> > gone
> > >> wrong. I think the stack trace you posted indicates that there is an
> > error
> > >> earlier in the RS logs that shows what is wrong with the compressor.
> > >> "Algorithm 'lzo' previously failed test". Look for when the test
> > happened.
> > >>
> > >>
> > >> Cheers,
> > >> Friso
> > >>
> > >>
> > >> On 28 mrt 2011, at 17:01, George P. Stathis wrote:
> > >>
> > >> Has anything changed with the way compression is handled in 0.90?
> > >>
> > >> I'm in the process of testing 0.90 CDH3B4 on my development machine
> (OS
> > >> X).
> > >> Up until 0.89 SNAPSHOTS, I have been able to re-use the same lzo JAR
> and
> > >> native libs that I compiled back in 0.20.x. With this latest upgrade,
> I
> > >> now
> > >> see this in the region server logs:
> > >>
> > >> 2011-03-28 10:54:51,787 ERROR
> > >> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
> > >> open
> > >> of region=monitors,,1287788998658.a6c3f7f6a927fda960a856c274b470c0.
> > >> java.io.IOException: Compression algorithm 'lzo' previously failed
> test.
> > >> at
> > >>
> > >>
> >
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:77)
> > >> at
> > >>
> > >>
> >
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2555)
> > >> at
> > >>
> > >>
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2544)
> > >> at
> > >>
> > >>
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532)
> > >> at
> > >>
> > >>
> >
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:262)
> > >> at
> > >>
> > >>
> >
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:94)
> > >> at
> > >>
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)
> > >> at
> > >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> at
> > >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >> at java.lang.Thread.run(Thread.java:680)
> > >>
> > >> Just want to make sure I'm not missing something before I start
> > >> re-compiling
> > >> those libs.
> > >>
> > >> -GS
> > >>
> > >
> > >
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message