ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-22628) YARN Shuffle Service Can't Be Found On Client-Only Nodes After New Cluster Install
Date Thu, 18 Jan 2018 18:27:03 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-22628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330921#comment-16330921
] 

Hudson commented on AMBARI-22628:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8613 (See [https://builds.apache.org/job/Ambari-trunk-Commit/8613/])
AMBARI-22628 - YARN Shuffle Service Can't Be Found On Client-Only Nodes (rlevas: [https://gitbox.apache.org/repos/asf?p=ambari.git&a=commit&h=15d8d1a1485ec5e5441c31566ef8a16632d6046f])
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py
* (edit) ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
* (edit) ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
* (edit) ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java


> YARN Shuffle Service Can't Be Found On Client-Only Nodes After New Cluster Install
> ----------------------------------------------------------------------------------
>
>                 Key: AMBARI-22628
>                 URL: https://issues.apache.org/jira/browse/AMBARI-22628
>             Project: Ambari
>          Issue Type: Bug
>    Affects Versions: 2.6.1
>            Reporter: Kishor Ramakrishnan
>            Assignee: Jonathan Hurley
>            Priority: Blocker
>             Fix For: 2.6.1
>
>         Attachments: AMBARI-22628.patch
>
>
> Installing a new cluster can create values in yarn-site.xml which have {{None}} specified
in the classpath for Spark
> {code:java}
> <property>
>       <name>yarn.nodemanager.aux-services.spark2_shuffle.classpath</name>
>       <value>/usr/hdp/None/spark2/aux/*</value>
>     </property>
>  <property>
>       <name>yarn.nodemanager.aux-services.spark_shuffle.classpath</name>
>       <value>/usr/hdp/None/spark/aux/*</value>
>     </property>
> <property>
>       <name>yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath</name>
>       <value>/usr/hdp/None/spark/hdpLib/*</value>
>     </property>
> {code}
> The cause for this is that YARN Clients on hosts without daemons never get a restart
command after the initial {{yarn-site.xml}}, and can never fill in the correct values. This
causes problems when jobs are run on these nodes:
> {code}
> 2017-12-04 10:16:41,789 INFO  service.AbstractService (AbstractService.java:noteFailure(272))
- Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed in
state INITED; cause: java.lang.ClassNotFoundException: org.apache.spark.network.yarn.YarnShuffleService
> java.lang.ClassNotFoundException: org.apache.spark.network.yarn.YarnShuffleService
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message