hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kozlov <ale...@cloudera.com>
Subject Re: Native libraries on OS X
Date Mon, 22 Feb 2010 21:03:41 GMT
Have you set java.library.path?  Add "-*Djava.library.path=$HOME**/**
hadoop-0.20.1/build/native/Mac_OS_X-x86_64-64/lib"* to
mapred.child.java.opts.

On Mon, Feb 22, 2010 at 12:51 PM, Derek Brown <derek@media6degrees.com>wrote:

> I was (seemingly) able to build the native libraries, but still get the
> "unable to load native-hadoop library" message at run-time (with a simple
> app of my own to read/write a gzip-compressed sequence file).
> Based on https://issues.apache.org/jira/browse/HADOOP-3659, "Patch to
> allow
> hadoop native to compile on Mac OS X", and other research, I did the
> following:
>
> $ cd hadoop-0.20.1/
> $ export
>
> LDFLAGS=-L/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Libraries
> $ patch -p0 -b < HADOOP-3659.patch
> $ cd src/native/
> $ # To regenerate the configure script:
> $ autoreconf -ifv
> $ cd ../..
> $ ant compile-native
> $ ls -l build/native/Mac_OS_X-x86_64-64/lib
> total 248
> -rw-r--r--  1 derek  staff  14007 Feb 22 13:26 Makefile
> lrwxr-xr-x  1 derek  staff     17 Feb 22 13:26 libhadoop.1.0.0.dylib@ ->
> libhadoop.1.dylib
> -rwxr-xr-x  1 derek  staff   8404 Feb 22 13:26 libhadoop.1.dylib*
> -rw-r--r--  1 derek  staff  84880 Feb 22 13:26 libhadoop.a
> lrwxr-xr-x  1 derek  staff     17 Feb 22 13:26 libhadoop.dylib@ ->
> libhadoop.1.dylib
> -rw-r--r--  1 derek  staff    900 Feb 22 13:26 libhadoop.la
>
> But the this still comes up when running my code:
>
> 10/02/22 15:10:29 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Exception in thread "main" java.lang.IllegalArgumentException: SequenceFile
> doesn't work with GzipCodec without native-hadoop code!
>
> I'm running my code as follows:
>
> $ hadoop jar my.jar com.media6.MyClass [args to MyClass here]
>
> I'm on OS X 10.5.8 Leopard, with an Intel Core 2 Duo MacBook.
>
> I didn't install any other libraries, which there are some references to in
> the Jira issue, since the build succeeded.
>
> hadoop.native.lib is true by default, but to be explicit, I added it to
> core-site.xml anyway.
>
> Thanks for any pointers,
> Derek
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message