hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phantom <ghostwhoowa...@gmail.com>
Subject Re: Minimal requirements for building libhdfs
Date Mon, 11 Jun 2007 20:54:41 GMT
So what I want is a C/C++ interface to write to HDFS ? What do I have to do
to achieve this ?

Thanks
A

On 6/11/07, Arun C Murthy <arunc@yahoo-inc.com> wrote:
>
> On Mon, Jun 11, 2007 at 11:16:55AM -0700, Phantom wrote:
> >I tried building with a 32 bit install of Java. Now I get the following
> >error w.r.t zlib. What is the workaround for me ?
> >
>
> So, I assume you want libhadoop rather than libhdfs...
>
> Sorry if my previous reply was vague: amd64 *is* x64 (aka
> AA64/IA-32e/EM64T etc.), so that isn't the problem.
>
> If you are not particular about using a 64-bit jvm, and Linux is your OS,
> I'd suggest you first grab the Linux-i386-32 pre-built libhadoop from the
> 0.13.0 release and try it...
>
> If that doesn't work for you please double-check your (preferred)
> 64/32-bit platform and ensure you have the relevant jvm and zlib/lzo libs.
>
> Arun
>
> >[exec] /usr/bin/make  all-recursive
> >    [exec] make[1]: Entering directory
> `/home/alakshman/FB-Projects/hadoop-
> >0.13.0/build/native/Linux-i386-32'
> >    [exec] Making all in src/org/apache/hadoop/io/compress/zlib
> >    [exec] make[2]: Entering directory
> `/home/alakshman/FB-Projects/hadoop-
> >0.13.0/build/native/Linux-i386-32/src/org/apache/hadoop/io/compress/zlib'
> >    [exec] if /bin/sh ../../../../../../../libtool --tag=CC
> --mode=compile
> >gcc -DHAVE_CONFIG_H -I.
> >-I/home/alakshman/FB-Projects/hadoop-0.13.0
> /src/native/src/org/apache/hadoop/io/compress/zlib
> >-I../../../../../../..  -I/home/alakshman/jdk1.5.0_06/include
> >-I/home/alakshman/jdk1.5.0_06/include/linux
> >-I/home/alakshman/FB-Projects/hadoop-0.13.0/src/native/src  -g -Wall
> -fPIC
> >-O2 -m32 -g -O2 -MT ZlibCompressor.lo -MD -MP -MF
> ".deps/ZlibCompressor.Tpo"
> >-c -o ZlibCompressor.lo
> >/home/alakshman/FB-Projects/hadoop-0.13.0
> /src/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c;
> >\
> >    [exec] then mv -f ".deps/ZlibCompressor.Tpo"
> >".deps/ZlibCompressor.Plo"; else rm -f ".deps/ZlibCompressor.Tpo"; exit
> 1;
> >fi
> >    [exec] mkdir .libs
> >    [exec]  gcc -DHAVE_CONFIG_H -I. -I/home/alakshman/FB-Projects/hadoop-
> >0.13.0/src/native/src/org/apache/hadoop/io/compress/zlib
> >-I../../../../../../.. -I/home/alakshman/jdk1.5.0_06/include
> >-I/home/alakshman/jdk1.5.0_06/include/linux
> >-I/home/alakshman/FB-Projects/hadoop-0.13.0/src/native/src -g -Wall -fPIC
> >-O2 -m32 -g -O2 -MT ZlibCompressor.lo -MD -MP -MF
> .deps/ZlibCompressor.Tpo
> >-c
> >/home/alakshman/FB-Projects/hadoop-0.13.0
> /src/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c
> >-fPIC -DPIC -o .libs/ZlibCompressor.o
> >    [exec] In file included from /usr/include/features.h:337,
> >    [exec]                  from /usr/include/stdio.h:28,
> >    [exec]                  from /home/alakshman/FB-Projects/hadoop-
> 0.13.0
> >/src/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c:24:
> >    [exec] /usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No such
> >file or directory
> >    [exec] make[2]: *** [ZlibCompressor.lo] Error 1
> >    [exec] make[2]: Leaving directory
> `/home/alakshman/FB-Projects/hadoop-
> >0.13.0/build/native/Linux-i386-32/src/org/apache/hadoop/io/compress/zlib'
> >    [exec] make[1]: *** [all-recursive] Error 1
> >    [exec] make[1]: Leaving directory
> `/home/alakshman/FB-Projects/hadoop-
> >0.13.0/build/native/Linux-i386-32'
> >    [exec] make: *** [all] Error 2
> >
> >Thanks
> >A
> >
> >On 6/11/07, Phantom <ghostwhoowalks@gmail.com> wrote:
> >>
> >>I ran the java org.apache.hadoop.util.Platform and it says that it is an
> >>AMD64. Your guess that I have an AMD64 bit install of Java is correct.
> Now
> >>is there a 64 bit install for x86 Linux ? I don't seem to find one on
> the
> >>Sun website. Does that mean I have to run this with 32 bit binaries for
> >>Java
> >>and for Lzo and zlib too ? Please advice.
> >>
> >>
> >>On 6/10/07, Arun C Murthy <arunc@yahoo-inc.com> wrote:
> >>>
> >>> There are 2 distinctive native components to hadoop:
> >>> a) libhdfs - Jni based C bindings for the hadoop DFS java libraries.
> >>> b) libhadoop - Native libraries for core-hadoop (for now we have lzo
> and
> >>> zlib compression libraries:
> >>> http://wiki.apache.org/lucene-hadoop/NativeHadoop as Doug pointed
> out).
> >>>
> >>> On Fri, Jun 08, 2007 at 04:14:04PM -0700, Phantom wrote:
> >>> >I am running into this wierd build problem - I am building this on
> >>> Fedora
> >>> >Linux x86 64 bit machine but the build is spitting out AMD64 library.
> >>> How
> >>> >can I fix this ? Here is the error from the build :
> >>> >
> >>>
> >>> Looking at the errors below 2 data-points:
> >>>
> >>> >    [exec] /bin/sh ../../../../../../../libtool --tag=CC --mode=link
> >>> gcc -g
> >>> >-Wall -fPIC -O2 -m64 -g -O2
> >>> -L/usr/local/jdk1.5.0_07/jre/lib/amd64/server
> >>> >-o libnativezlib.la   ZlibCompressor.lo ZlibDecompressor.lo -ldl
> -ljvm
> >>> -ljvm
> >>>
> >>> and
> >>>
> >>> >    [exec] Making all in src/org/apache/hadoop/io/compress/lzo
> >>> >    [exec] make[2]: Entering directory
> >>> `/home/alakshman/FB-Projects/hadoop-
> >>>
> >>>0.13.0
> /build/native/Linux-amd64-64/src/org/apache/hadoop/io/compress/lzo'
> >>>
> >>>
> >>> (emphasis on -L/usr/local/jdk1.5.0_07/jre/lib/amd64/server and 0.13.0
> >>> /build/native/Linux-amd64-64/src/org/apache/hadoop/io/compress/lzo)
> >>>
> >>> lead me to suspect that you have a amd64 build of the jvm installed...
> >>> could you run org.apache.hadoop.util.PlatformName and check? Also
> double
> >>> check the 'JAVA_HOME' env. variable...
> >>>
> >>> Arun
> >>>
> >>> >
> >>> >BUILD FAILED
> >>> >/home/alakshman/FB-Projects/hadoop-0.13.0/build.xml:285: exec
> returned:
> >>> 2
> >>> >
> >>> >
> >>> >On 6/8/07, Doug Cutting <cutting@apache.org> wrote:
> >>> >>
> >>> >>Phantom wrote:
> >>> >>> What are minimal requirements on my Linux machine for building
> >>> libhdfs ?
> >>> >>On
> >>> >>> my Linux box I do not seem to have jni.h and what are the other
> >>> binaries
> >>> >>I
> >>> >>> need for this to work ? Could someone please tell me what is
the
> >>> easiest
> >>> >>> way
> >>> >>> to get this done ?
> >>> >>
> >>> >>The following wiki page should help:
> >>> >>
> >>> >>http://wiki.apache.org/lucene-hadoop/NativeHadoop
> >>> >>
> >>> >>Doug
> >>> >>
> >>>
> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message