hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peilin" <xpei...@gmail.com>
Subject A problem in developing custom InputFormat with .SO
Date Tue, 13 Aug 2013 08:11:52 GMT

Im a new hadoop dev.

I got a problem when I develop an costum InputFormat.class

I need this InputFormat.class pull files from other FS, not HDFS, and this
FS is coded by c++

So I package the APIs in an .so file, then call these APIs in java

I write the java file and package all .class, .so files in an JAR.

Then I submit this JAR and other MR files to the hadoop server(I submit the
Job with Streaming)


There are some problems:

         I submit these files successfully, but when the task start, there
are exceptions:

         This means my class is invalid? 

         I don't know how to solve it.


         Another problem:

                   I don't know how to set the LD_LIBRARY_PATH in hadoop


         Who can help me.


         Thanks very much.



  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message