hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Adding external JAR with Native Libraries (UBUNTU)
Date Sun, 21 Jun 2015 04:10:59 GMT
Good luck! :-)

On Sat, Jun 20, 2015 at 11:24 PM, Behroz Sikander <behroz89@gmail.com> wrote:
> Hi,
> Thanks Edward. You were right :). I had 4 BSP tasks and the following
> solution worked for me. Thanks.
>
> *hama-site.xml*
> </property>
>          <property>
>                  <name>bsp.child.java.opts</name>
>
>  <value>-Djava.library.path=/home/behroz/Documents/cplex_setup/cplex/bin/x86-64_linux/</value>
>                  <description>Java opts for the groom server child
> processes.
>                  The following symbol, if present, will be interpolated:
> @taskid@ is replaced
>                  by current TaskID. Any other occurrences of '@' will go
> unchanged.
>                  For example, to enable verbose gc logging to a file named
> for the taskid in
>                  /tmp and to set the heap maximum to be a gigabyte, pass a
> 'value' of:
>                         -Xmx1024m -verbose:gc -Xloggc:/tmp/@taskid@.gc
>                  The configuration variable bsp.child.ulimit can be used to
> control the
>                  maximum virtual memory of the child processes.
>                  </description>
>         </property>
>
> On Sat, Jun 20, 2015 at 4:33 AM, Behroz Sikander <behroz89@gmail.com> wrote:
>
>> I tried to add HAMA_OPT in hama-env.sh but I get the same error. I did the
>> following steps
>> 1- Stopped Hama
>> 2- Stopped Hadoop
>> 3- Modified the file
>> 4- Started Hadoop
>> 5- Started Hama
>> 6- Ran the ./bin/hama jar command
>>
>> *Content of hama-env.sh*
>> # The java implementation to use.  Required.
>> export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/
>>
>> export
>> HAMA_OPTS="-Djava.library.path=/home/behroz/Documents/cplex_setup/cplex/bin/x86-64_linux"
>>
>> On Sat, Jun 20, 2015 at 4:25 AM, Edward J. Yoon <edwardyoon@apache.org>
>> wrote:
>>
>>> And, if "-Djava.library.path=$CPLEX_HOME" is needed for every BSP
>>> tasks, you should use bsp.child.java.opts property. :-)
>>>
>>>   <property>
>>>     <name>bsp.child.java.opts</name>
>>>     <value>-Xmx2048m</value>
>>>     <description>Java opts for the groom server child processes.
>>>     The following symbol, if present, will be interpolated: @taskid@ is
>>> replaced
>>>     by current TaskID. Any other occurrences of '@' will go unchanged.
>>>     For example, to enable verbose gc logging to a file named for the
>>> taskid in
>>>     /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of:
>>>           -Xmx1024m -verbose:gc -Xloggc:/tmp/@taskid@.gc
>>>     The configuration variable bsp.child.ulimit can be used to control the
>>>     maximum virtual memory of the child processes.
>>>     </description>
>>>   </property>
>>>
>>> On Sat, Jun 20, 2015 at 11:22 AM, Edward J. Yoon <edwardyoon@apache.org>
>>> wrote:
>>> > Please try to add below line into hama-env.sh file:
>>> >
>>> > export HAMA_OPTS="-Djava.library.path=/your/lib/path/"
>>> >
>>> > then, option will be added automatically when you use
>>> > '${HAMA_HOME}/bin/hama jar' command:
>>> > --
>>> >
>>> > Edwardui-MacBook-Pro:hama-trunk edwardyoon$ bin/hama jar
>>> >
>>> > -Djava.library.path=/your/lib/path/
>>> > -Dhama.log.dir=/Users/edwardyoon/workspace/hama-trunk/bin/../logs
>>> > -Dhama.log.file=hama.log
>>> > -Dhama.home.dir=/Users/edwardyoon/workspace/hama-trunk/bin/..
>>> > -Dhama.id.str= -Dhama.root.logger=INFO,console
>>> > -Dhama.policy.file=hama-policy.xml
>>> >
>>> > Usage: hama jar <jar> [mainClass] args...
>>> >
>>> > On Sat, Jun 20, 2015 at 10:03 AM, Behroz Sikander <behroz89@gmail.com>
>>> wrote:
>>> >> I tried to set $HAMA_OPT in /etc/environment (a system file to store
>>> >> environment variables) but it didn't work. Then I modified the
>>> >> $HAMA_HOME/bin/hama but it also didn't work. I added just the following
>>> >> line in $HAMA_HOME/bin/hama file.
>>> >>
>>> >> *HAMA_OPTS="$HAMA_OPTS -Djava.library.path=$CPLEX_HOME"*
>>> >>
>>> >> *Note: $CPLEX_HOME is also defined in /etc/environment*
>>> >>
>>> >> Regards,
>>> >> Behroz
>>> >>
>>> >> On Sat, Jun 20, 2015 at 2:04 AM, JĂșlio Pires <juliocspires@gmail.com>
>>> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> You tried to set the HAMA_OPTS in $HAMA_HOME/bin/hama or
>>> >>> $HAMA_HOME/conf/hama-env.sh?
>>> >>>
>>> >>> Thanks!
>>> >>>
>>> >>> 2015-06-19 20:13 GMT-03:00 Behroz Sikander <behroz89@gmail.com>:
>>> >>>
>>> >>> > Hi,
>>> >>> > I recently wrote an algorithm on Hama which uses a third party
>>> library
>>> >>> > (CPLEX). This library comes with a JAR file and folder with
Native
>>> >>> > Libraries. I am using Ubuntu and my Hama version is 0.6.4.
>>> >>> >
>>> >>> > In eclipse everything is fine. I added the native library folder
in
>>> >>> "Native
>>> >>> > Library location" option of JAR file in eclipse (Project properties
>>> >
>>> >>> Build
>>> >>> > Path > libraries > Select CPLEX > open dropdown and
set the
>>> location) and
>>> >>> > my code worked fine. Now, I am trying to deploy the code on
the
>>> cluster.
>>> >>> I
>>> >>> > tried on a single machine but I get the following exception
>>> >>> >
>>> >>> > *java.lang.UnsatisfiedLinkError: ilog.cplex.Cplex.CPXopenCPLEX([I)J*
>>> >>> > *attempt_201506192258_0001_000001_0: at
>>> >>> > ilog.cplex.Cplex.CPXopenCPLEX(Native Method)*
>>> >>> > *attempt_201506192258_0001_000001_0: at
>>> >>> > ilog.cplex.CplexI.init(CplexI.java:6608)*
>>> >>> > *attempt_201506192258_0001_000001_0: at
>>> >>> > ilog.cplex.CplexI.<init>(CplexI.java:629)*
>>> >>> >
>>> >>> > The above error is occuring because it was not able to locate
the
>>> native
>>> >>> > library of CPLEX jar (I placed the CPLEX jar in $HAMA_HOME/lib/
).
>>> I used
>>> >>> > the following command to run my jar
>>> >>> > *./bin/hama jar myjar.jar <inputs>*
>>> >>> >
>>> >>> >
>>> >>> > I tried to set the HAMA_OPTS in /etc/envirnment but still I
get the
>>> same
>>> >>> > error.
>>> >>> > *export
>>> >>> >
>>> >>> >
>>> >>>
>>> HAMA_OPTS="-Djava.library.path=/home/behroz/Documents/cplex_setup/cplex/bin/x86-64_linux"*
>>> >>> >
>>> >>> > Without HAMA if I run the following command I get no error
and
>>> everything
>>> >>> > works fine
>>> >>> > *java
>>> >>> >
>>> >>> >
>>> >>>
>>> -Djava.library.path=/home/behroz/Documents/cplex_setup/cplex/bin/x86-64_linux*
>>> >>> >
>>> >>> >
>>> >>> > Can you please guide me how to resolve this issue ?
>>> >>> >
>>> >>> > Regards,
>>> >>> > Behroz Sikander
>>> >>> >
>>> >>>
>>> >
>>> >
>>> >
>>> > --
>>> > Best Regards, Edward J. Yoon
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>>
>>
>>



-- 
Best Regards, Edward J. Yoon

Mime
View raw message