ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 27348: After adding hdp.version in yarn-site.xml Distributed Shell is failing
Date Wed, 29 Oct 2014 21:50:51 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27348/
-----------------------------------------------------------

(Updated Oct. 29, 2014, 9:50 p.m.)


Review request for Ambari, Dmytro Sen, Mahadev Konar, Nate Cole, Sumit Mohanty, Srimanth Gunturi,
Sid Wagle, and Vinod Kumar Vavilapalli.


Bugs: AMBARI-8028
    https://issues.apache.org/jira/browse/AMBARI-8028


Repository: ambari


Description
-------

After adding hdp.version in yarn-site.xml Distributed Shell is failing 
```
 /usr/hdp/current/hadoop-yarn-client/bin/yarn  org.apache.hadoop.yarn.applications.distributedshell.Client
-jar /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar -shell_command
pwd
```

Failing with classpath issue:
```
Diagnostics: Exception from container-launch.
Container id: container_1414595622703_0039_02_000001
Exit code: 1
Exception message: /grid/1/yarn/local/usercache/hrt_qa/appcache/application_1414595622703_0039/container_1414595622703_0039_02_000001/launch_container.sh:
line 9: $CLASSPATH:./*:$HADOOP_CONF_DIR:/usr/hdp/${hdp.version}/hadoop-client/*:/usr/hdp/${hdp.version}/hadoop-client/lib/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/lib/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/lib/*:./log4j.properties:
bad substitution

Stack trace: ExitCodeException exitCode=1: /grid/1/yarn/local/usercache/hrt_qa/appcache/application_1414595622703_0039/container_1414595622703_0039_02_000001/launch_container.sh:
line 9: $CLASSPATH:./*:$HADOOP_CONF_DIR:/usr/hdp/${hdp.version}/hadoop-client/*:/usr/hdp/${hdp.version}/hadoop-client/lib/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/lib/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/lib/*:./log4j.properties:
bad substitution

        at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
        at org.apache.hadoop.util.Shell.run(Shell.java:455)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:206)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:301)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1

```
Though in secure this not reported as front-end error, AM  logs show classpath issue there
also
```
Container: container_1414601160300_0005_01_000001 on karam-tez-sec-exp1-3.cs1cloud.internal_45454
===================================================================================================
LogType: AppMaster.stderr
LogUploadTime: 29-Oct-2014 16:51:42
LogLength: 996
Log Contents:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnException
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2570)
        at java.lang.Class.getMethod0(Class.java:2813)
        at java.lang.Class.getMethod(Class.java:1663)
        at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
```


Diffs
-----

  ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
defdbb5 

Diff: https://reviews.apache.org/r/27348/diff/


Testing
-------

Made the property change on an HDP 2.2 cluster (single node), restarted YARN, MR, HIVE, OOZIE,
ZOOKEEPER, and then ran a service check for the same items, all of which passed.

mvn clean test
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24:41.350s
[INFO] Finished at: Wed Oct 29 14:09:52 PDT 2014
[INFO] Final Memory: 65M/1041M
[INFO] ------------------------------------------------------------------------


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message