Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of peter.s.naudus@gmail.com
 designates 209.85.216.169 as permitted sender)
Received-SPF: pass (google.com: domain of peter.s.naudus@gmail.com designates
 10.224.73.13 as permitted sender) client-ip=10.224.73.13;
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes
To: user@hbase.apache.org, Stack <stack@duboce.net>
Subject: Re: java.io.IOException: Compression algorithm 'snappy' previously
 failed test
Reply-To: pnaudus@dataraker.com
References: <op.waescyeemrx7t6@localhost.localdomain>
 <CADcMMgENW4CceT-W3E4F3GnGsFhSJZ6_e95Ldkh+eXfpWTzpRA@mail.gmail.com>
Date: Wed, 29 Feb 2012 12:52:33 -0500
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: "Peter Naudus" <pnaudus@dataraker.com>
Organization: DataRaker
Message-ID: <op.wagbxvrpmrx7t6@localhost.localdomain>
In-Reply-To: 
 <CADcMMgENW4CceT-W3E4F3GnGsFhSJZ6_e95Ldkh+eXfpWTzpRA@mail.gmail.com>
User-Agent: Opera Mail/11.61 (Linux)

Thanks for your help :)

To make sure I manually set LD_LIBRARY_PATH, LIBRARY_PATH, and  
HBASE_LIBRARY_PATH

bash-3.2$ export  
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/dataraker/software/cdh4/hadoop-0.23.0-cdh4b1/lib/native
bash-3.2$ export  
LIBRARY_PATH=$LIBRARY_PATH:/opt/dataraker/software/cdh4/hadoop-0.23.0-cdh4b1/lib/native
bash-3.2$ export  
HBASE_LIBRARY_PATH=/opt/dataraker/software/cdh4/hadoop-0.23.0-cdh4b1/lib/native

But running the compression test failed with "native snappy library not  
available"

bash-3.2$ ./hbase org.apache.hadoop.hbase.util.CompressionTest  
file:///tmp/test.txt snappy
log4j:WARN No appenders could be found for logger  
(org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for  
more info.
Exception in thread "main" java.lang.RuntimeException: native snappy  
library not available
	at  
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:121)
	at  
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:104)
	at  
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:118)
	at  
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
	at  
org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:588)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:178)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:150)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:140)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:104)
	at  
org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:108)
	at  
org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:137)

I verified that libsnappy is indeed, installed

bash-3.2$ ls -al $HBASE_LIBRARY_PATH
total 1412
drwxr-xr-x 2 1106 592   4096 Feb 11 01:06 .
drwxr-xr-x 3 1106 592   4096 Feb 11 01:06 ..
-rw-r--r-- 1 1106 592 616862 Feb 11 01:06 libhadoop.a
-rwxr-xr-x 1 1106 592   1051 Feb 11 01:06 libhadoop.la
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libhadoop.so ->  
libhadoop.so.1.0.0
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libhadoop.so.1 ->  
libhadoop.so.1.0.0
-rwxr-xr-x 1 1106 592 340361 Feb 11 01:06 libhadoop.so.1.0.0
-rw-r--r-- 1 1106 592 184418 Feb 11 01:06 libhdfs.a
-rwxr-xr-x 1 1106 592   1034 Feb 11 01:06 libhdfs.la
lrwxrwxrwx 1 1106 592     16 Feb 27 18:12 libhdfs.so -> libhdfs.so.0.0.0
lrwxrwxrwx 1 1106 592     16 Feb 27 18:12 libhdfs.so.0 -> libhdfs.so.0.0.0
-rwxr-xr-x 1 1106 592 125455 Feb 11 01:06 libhdfs.so.0.0.0
-rw-r--r-- 1 1106 592  37392 Feb 11 01:06 libsnappy.a
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libsnappy.so ->  
libsnappy.so.1.1.1
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libsnappy.so.1 ->  
libsnappy.so.1.1.1
-rw-r--r-- 1 1106 592  26824 Feb 11 01:06 libsnappy.so.1.1.1

Just for grins and giggles I re-ran this as root

In addition to the Exception mentioned above, I also got following Warning:
	WARN util.NativeCodeLoader: Unable to load native-hadoop library for your  
platform... using builtin-java classes where applicable

Any ideas?

On Tue, 28 Feb 2012 20:02:38 -0500, Stack <stack@duboce.net> wrote:

> On Tue, Feb 28, 2012 at 1:52 PM, Peter Naudus <pnaudus@dataraker.com>  
> wrote:
>> What else can I do to fix / diagnose this problem?
>>
>
> Does our little compression tool help?
> http://hbase.apache.org/book.html#compression.test
>
> St.Ack


--