ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-21872) Deploys failing with MR service check failure due to missing mapreduce.tar.gz
Date Tue, 05 Sep 2017 16:20:00 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153906#comment-16153906
] 

Hudson commented on AMBARI-21872:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7996 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7996/])
AMBARI-21872. Deploys failing with MR service check failure due to (aonishuk: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=93011b2600d0e5f3d05f1b8d317239c0e34662f5])
* (edit) ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py


> Deploys failing with MR service check failure due to missing mapreduce.tar.gz
> -----------------------------------------------------------------------------
>
>                 Key: AMBARI-21872
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21872
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Andrew Onischuk
>            Assignee: Andrew Onischuk
>            Priority: Blocker
>             Fix For: 3.0.0
>
>         Attachments: AMBARI-21872.patch
>
>
> **Problem:**
>     
>     
>     
>     2017-08-30 01:31:33,674 - ExecuteHadoop['jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-3.*.jar
wordcount /user/ambari-qa/mapredsmokeinput /user/ambari-qa/mapredsmokeoutput'] {'bin_dir':
'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin:/usr/hdp/current/hadoop-yarn-client/bin',
'conf_dir': '/usr/hdp/current/hadoop-client/conf', 'logoutput': True, 'try_sleep': 5, 'tries':
1, 'user': 'ambari-qa'}
>     2017-08-30 01:31:33,675 - Execute['hadoop --config /usr/hdp/current/hadoop-client/conf
jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-3.*.jar wordcount /user/ambari-qa/mapredsmokeinput
/user/ambari-qa/mapredsmokeoutput'] {'logoutput': True, 'try_sleep': 5, 'environment': {},
'tries': 1, 'user': 'ambari-qa', 'path': ['/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin:/usr/hdp/current/hadoop-yarn-client/bin']}
>     ######## Hortonworks #############
>     This is MOTD message, added for testing in qe infra
>     17/08/30 01:31:35 INFO client.RMProxy: Connecting to ResourceManager at ctr-e134-1499953498516-113587-01-000003.hwx.site/172.27.31.9:8050
>     java.io.FileNotFoundException: File does not exist: hdfs://ctr-e134-1499953498516-113587-01-000003.hwx.site:8020/hdp/apps/3.0.0.0-229/mapreduce/mapreduce.tar.gz
>     	at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:137)
>     	at org.apache.hadoop.fs.AbstractFileSystem.resolvePath(AbstractFileSystem.java:480)
>     	at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2191)
>     	at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2187)
>     	at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
>     	at org.apache.hadoop.fs.FileContext.resolve(FileContext.java:2193)
>     	at org.apache.hadoop.fs.FileContext.resolvePath(FileContext.java:598)
>     	at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:465)
>     	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:150)
>     	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1344)
>     
> **Investigation:**  
> After starting History Server, (which includes adding mapreduce.tar.gz to
> HDFS), it complains about HDFS_CLIENT not present.
>     
>     
>     2017-09-01 21:16:04,194 - Called copy_to_hdfs tarball: mapreduce
>     2017-09-01 21:16:04,194 - Stack Feature Version Info: Cluster Stack=3.0, Command
Stack=None, Command Version=3.0.0.0-229 -> 3.0.0.0-229
>     2017-09-01 21:16:04,194 - Tarball version was calcuated as 3.0.0.0-229. Use Command
Version: True
>     2017-09-01 21:16:04,194 - HDFS_CLIENT is not installed on the host. Skip copying
/usr/hdp/3.0.0.0-229/hadoop/mapreduce.tar.gz
>     
> In Ambari-2.5.2 history server host does not have MR client/Hadoop client and
> has MR service check passed.
> **Solution:**  
> Seems like Ambari checks if a certain component is installed on the cluster
> (like HDFS_CLIENT). However not only HDFS_CLIENT but namenode, datanode,
> nfsgateway, journalnode etc.  
> would include tar.gz. We have to check instead if cluster has a service, but
> not that host has a component.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message