hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henning Blohm <henning.bl...@zfabrik.de>
Subject Setting extra classpath and library path for Yarn M/R Application
Date Wed, 30 Mar 2016 10:54:35 GMT

I am having a hard time figuring out how to add command line options, an 
extra class path and library path to a map reduce application running on 
Hadoop 2.6.0 (CDH 5.6).

Unfortunately this application is using a native library with a 
pre-installed wrapper JAR that is installed on all data nodes. So what I 
am trying to achieve is to pass an LD_LIBRARY_PATH and an extra 
classpath to the application runner.

Previously, I used

         <value>-XX:-UseSplitVerifier -Xmx500M -XX:MaxPermSize=256M 

in the client config file for command line options and set

export HADOOP_CLASSPATH=/pathToWrapperJar
export JAVA_LIBRARY_PATH=/nativepath/

in my Hadoop environment.  None of this seems to work with Hadoop 2.6.0 

Here is what I have now:

I managed to add to LD_LIBRARY_PATH by specifying


         <value>-Da=1 -Db=2</value>

in the client config file. I can tell that by cat'ing the 
launch_container.sh while it is present.

I do however fail to add anything to the application's class path. Plus 
I would rather not specify the native library path and wrapper jar in 
the client config but rather globally.

It would be great, if somebody can give me a hint first and foremost on 
the class path question and secondly some orientation on whether using 
the new configuration properties is the way to go.


View raw message