hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Naudus" <pnau...@dataraker.com>
Subject Re: java.io.IOException: Compression algorithm 'snappy' previously failed test
Date Wed, 29 Feb 2012 17:52:33 GMT
Thanks for your help :)

To make sure I manually set LD_LIBRARY_PATH, LIBRARY_PATH, and  
HBASE_LIBRARY_PATH

bash-3.2$ export  
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/dataraker/software/cdh4/hadoop-0.23.0-cdh4b1/lib/native
bash-3.2$ export  
LIBRARY_PATH=$LIBRARY_PATH:/opt/dataraker/software/cdh4/hadoop-0.23.0-cdh4b1/lib/native
bash-3.2$ export  
HBASE_LIBRARY_PATH=/opt/dataraker/software/cdh4/hadoop-0.23.0-cdh4b1/lib/native

But running the compression test failed with "native snappy library not  
available"

bash-3.2$ ./hbase org.apache.hadoop.hbase.util.CompressionTest  
file:///tmp/test.txt snappy
log4j:WARN No appenders could be found for logger  
(org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for  
more info.
Exception in thread "main" java.lang.RuntimeException: native snappy  
library not available
	at  
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:121)
	at  
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:104)
	at  
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:118)
	at  
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
	at  
org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:588)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:178)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:150)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:140)
	at  
org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:104)
	at  
org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:108)
	at  
org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:137)

I verified that libsnappy is indeed, installed

bash-3.2$ ls -al $HBASE_LIBRARY_PATH
total 1412
drwxr-xr-x 2 1106 592   4096 Feb 11 01:06 .
drwxr-xr-x 3 1106 592   4096 Feb 11 01:06 ..
-rw-r--r-- 1 1106 592 616862 Feb 11 01:06 libhadoop.a
-rwxr-xr-x 1 1106 592   1051 Feb 11 01:06 libhadoop.la
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libhadoop.so ->  
libhadoop.so.1.0.0
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libhadoop.so.1 ->  
libhadoop.so.1.0.0
-rwxr-xr-x 1 1106 592 340361 Feb 11 01:06 libhadoop.so.1.0.0
-rw-r--r-- 1 1106 592 184418 Feb 11 01:06 libhdfs.a
-rwxr-xr-x 1 1106 592   1034 Feb 11 01:06 libhdfs.la
lrwxrwxrwx 1 1106 592     16 Feb 27 18:12 libhdfs.so -> libhdfs.so.0.0.0
lrwxrwxrwx 1 1106 592     16 Feb 27 18:12 libhdfs.so.0 -> libhdfs.so.0.0.0
-rwxr-xr-x 1 1106 592 125455 Feb 11 01:06 libhdfs.so.0.0.0
-rw-r--r-- 1 1106 592  37392 Feb 11 01:06 libsnappy.a
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libsnappy.so ->  
libsnappy.so.1.1.1
lrwxrwxrwx 1 1106 592     18 Feb 27 18:12 libsnappy.so.1 ->  
libsnappy.so.1.1.1
-rw-r--r-- 1 1106 592  26824 Feb 11 01:06 libsnappy.so.1.1.1

Just for grins and giggles I re-ran this as root

In addition to the Exception mentioned above, I also got following Warning:
	WARN util.NativeCodeLoader: Unable to load native-hadoop library for your  
platform... using builtin-java classes where applicable

Any ideas?

On Tue, 28 Feb 2012 20:02:38 -0500, Stack <stack@duboce.net> wrote:

> On Tue, Feb 28, 2012 at 1:52 PM, Peter Naudus <pnaudus@dataraker.com>  
> wrote:
>> What else can I do to fix / diagnose this problem?
>>
>
> Does our little compression tool help?
> http://hbase.apache.org/book.html#compression.test
>
> St.Ack


--

Mime
View raw message