ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-8028) After adding hdp.version in yarn-site.xml Distributed Shell is failing
Date Wed, 13 May 2015 07:43:00 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541535#comment-14541535
] 

ASF GitHub Bot commented on AMBARI-8028:
----------------------------------------

Github user RPCMoritz commented on the pull request:

    https://github.com/apache/incubator-zeppelin/pull/65#issuecomment-101553576
  
    While this was a decent effort (it actually made my classpath-related error almost readable,
where previously it was three screens worth of text), there is still an issue with classpath's
lack of simplicity. In partcular, it pulls in (without any obvious documentation) many different
classpaths, some of which may depend on environment variables. I'm currently (Zeppelin master,
HDP 2.2w/ spark tech preview on yarn) getting "bad substitution" errors from launch_container.sh:27,
an issue similar to Ambari-8028 (https://issues.apache.org/jira/browse/AMBARI-8028) and which
I suspect is down to remaining references to ${hdp.version} (and potentially $PWD, $HADOOP_CONF_DIR),
which I'm chasing down now.
    
    I think it might be a good idea to move the classpath over to zeppelin-site.xml in any
case, since over the long term managing the classpath via shell scripts is going to be painful
and not user-friendly. Especially the CLASSPATH+= makes identifying the issue I encountered
very difficult.


> After adding hdp.version in yarn-site.xml Distributed Shell is failing
> ----------------------------------------------------------------------
>
>                 Key: AMBARI-8028
>                 URL: https://issues.apache.org/jira/browse/AMBARI-8028
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 1.7.0
>            Reporter: Alejandro Fernandez
>            Assignee: Alejandro Fernandez
>            Priority: Blocker
>             Fix For: 1.7.0
>
>         Attachments: AMBARI-8028.patch
>
>
> After adding hdp.version in yarn-site.xml Distributed Shell is failing 
> {code}
>  /usr/hdp/current/hadoop-yarn-client/bin/yarn  org.apache.hadoop.yarn.applications.distributedshell.Client
-jar /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar -shell_command
pwd
> {code}
> Failing with classpath issue:
> {code}
> Diagnostics: Exception from container-launch.
> Container id: container_1414595622703_0039_02_000001
> Exit code: 1
> Exception message: /grid/1/yarn/local/usercache/hrt_qa/appcache/application_1414595622703_0039/container_1414595622703_0039_02_000001/launch_container.sh:
line 9: $CLASSPATH:./*:$HADOOP_CONF_DIR:/usr/hdp/${hdp.version}/hadoop-client/*:/usr/hdp/${hdp.version}/hadoop-client/lib/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/lib/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/lib/*:./log4j.properties:
bad substitution
> Stack trace: ExitCodeException exitCode=1: /grid/1/yarn/local/usercache/hrt_qa/appcache/application_1414595622703_0039/container_1414595622703_0039_02_000001/launch_container.sh:
line 9: $CLASSPATH:./*:$HADOOP_CONF_DIR:/usr/hdp/${hdp.version}/hadoop-client/*:/usr/hdp/${hdp.version}/hadoop-client/lib/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/lib/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/lib/*:./log4j.properties:
bad substitution
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>         at org.apache.hadoop.util.Shell.run(Shell.java:455)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
>         at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:206)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:301)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1
> {code}
> Though in secure this not reported as front-end error, AM  logs show classpath issue
there also
> {code}
> Container: container_1414601160300_0005_01_000001 on karam-tez-sec-exp1-3.cs1cloud.internal_45454
> ===================================================================================================
> LogType: AppMaster.stderr
> LogUploadTime: 29-Oct-2014 16:51:42
> LogLength: 996
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnException
>         at java.lang.Class.getDeclaredMethods0(Native Method)
>         at java.lang.Class.privateGetDeclaredMethods(Class.java:2570)
>         at java.lang.Class.getMethod0(Class.java:2813)
>         at java.lang.Class.getMethod(Class.java:1663)
>         at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
>         at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message