hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Failing to get Snappy working (hours now - native snappy library not available) - Help?
Date Wed, 03 Apr 2013 20:46:41 GMT
Thanks for the reply, Jeff.
You can point HBASE_LIBRARY_PATH at the .so files.

>From bin/hbase:

#   HBASE_LIBRARY_PATH  HBase additions to JAVA_LIBRARY_PATH for adding
#                    native libaries.


On Wed, Apr 3, 2013 at 1:35 PM, Jeff Blaine <jblaine@kickflop.net> wrote:

> On 4/3/2013 4:25 PM, Ted Yu wrote:
>
>> Looking at http://hbase.apache.org/book.**html#snappy.compression<http://hbase.apache.org/book.html#snappy.compression>,
it seems
>> that there is no libsnappy.a in your native directory.
>>
>
> Hi Ted,
>
> libsnappy.a is an "archive" style static library for inserting
> into an executable at compile + link time. It's a bundle of
> object files.
>
> The .so files are dynamically loadable libraries.
>
> I'm pretty positive HBase does not care an ounce about any
> .a files.
>
> That page also states the shared (not static) library
> expectancy:
>
>     You will find the snappy library file under the .libs
>     directory from your Snappy build. The file is called
>     libsnappy.so.1.x.x where 1.x.x is the version of the snappy
>     code you are building. You can either copy this file into
>     your hbase directory under libsnappy.so name, or simply
>     create a symbolic link to it.
>
> This reply *DID* however help me realize that the instructions
> are telling me to put the libraries (or links) in the *HBASE*
> lib/native/Linux-etcetc... directory. Not *HADOOP* where I have
> them!
>
> Off to try that.
>
>  On Wed, Apr 3, 2013 at 1:21 PM, Jeff Blaine <jblaine@kickflop.net> wrote:
>>
>>  Hello,
>>>
>>> I cannot see what is amiss with the following configuration. I have
>>> followed, to the best of my knowledge, the instructions to a "T".
>>>
>>> If anyone could shed some light, I would be grateful. All HBase
>>> hosts have the exact same configuration shown below and non-SNAPPY
>>> table operations work fine...
>>>
>>> $ id
>>> uid=30000(hadoop) gid=30000(hadoop) groups=30000(hadoop)
>>> $ uname -i
>>> x86_64
>>> $ pwd
>>> /hbase-0.92.2
>>> $ grep HBASE_LIBRARY_PATH conf/hbase-env.sh
>>> export HBASE_LIBRARY_PATH="/hadoop-1.****0.4/lib/native/Linux-amd64-**
>>> 64"
>>> $ cd /hadoop-1.0.4/lib/native/****Linux-amd64-64
>>>
>>> $ ls -l
>>> total 1456
>>> -rw-rw-r-- 1 hadoop root   405228 Oct  3 05:14 libhadoop.a
>>> -rw-rw-r-- 1 hadoop root      878 Oct  3 05:14 libhadoop.la
>>> -rw-rw-r-- 1 hadoop root   219217 Oct  3 05:14 libhadoop.so
>>> -rw-rw-r-- 1 hadoop root   219217 Oct  3 05:14 libhadoop.so.1
>>> -rw-rw-r-- 1 hadoop root   219217 Oct  3 05:14 libhadoop.so.1.0.0
>>> -rw-rw-r-- 1 hadoop ubuntu 136297 Apr  3 19:46 libsnappy.so
>>> -rw-rw-r-- 1 hadoop ubuntu 136297 Apr  3 19:47 libsnappy.so.1
>>> -rw-rw-r-- 1 hadoop ubuntu 136297 Apr  3 19:46 libsnappy.so.1.1.4
>>> $ /hbase-0.92.2/bin/hbase shell
>>> HBase Shell; enter 'help<RETURN>' for list of supported commands.
>>> Type "exit<RETURN>" to leave the HBase Shell
>>> Version 0.92.2, r1379292, Fri Aug 31 13:13:53 UTC 2012
>>>
>>> hbase(main):001:0> create 'non', 'cf'
>>> 0 row(s) in 4.0720 seconds
>>>
>>> hbase(main):002:0> create 'comp', { NAME => 'foo', COMPRESSION =>
>>> 'SNAPPY'
>>> }
>>> [ The command above never returns ]
>>> ^C
>>> $
>>>
>>> Logs show:
>>>
>>> 2013-04-03 18:07:11,992 WARN org.apache.hadoop.io.compress.**
>>> **snappy.LoadSnappy:
>>>
>>> Snappy native library not loaded
>>> java.io.IOException: java.lang.RuntimeException: native snappy library
>>> not
>>> available
>>>
>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message