hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anubhav Dhoot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2624) Resource Localization fails on a cluster due to existing cache directories
Date Thu, 02 Oct 2014 04:53:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156079#comment-14156079
] 

Anubhav Dhoot commented on YARN-2624:
-------------------------------------

The fix addresses the scenario moving from  pre node manager recovery to turning on node manager
recovery. As per  YARN-1338 the directories are not cleaned up inorder to preserve running
containers. But uniqueNumberGenerator will not know about preexisting directories which were
normally deleted on NM startup and are unknown to recovery enabled NM. In this case we still
want directory cleanup to happen.

> Resource Localization fails on a cluster due to existing cache directories
> --------------------------------------------------------------------------
>
>                 Key: YARN-2624
>                 URL: https://issues.apache.org/jira/browse/YARN-2624
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.5.1
>            Reporter: Anubhav Dhoot
>            Assignee: Anubhav Dhoot
>            Priority: Blocker
>         Attachments: YARN-2624.001.patch, YARN-2624.001.patch
>
>
> We have found resource localization fails on a cluster with following error in certain
cases.
> {noformat}
> INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
Failed to download rsrc { { hdfs://<blahhostname>:8020/tmp/hive-hive/hive_2014-09-29_14-55-45_184_6531377394813896912-12/-mr-10004/95a07b90-2448-48fc-bcda-cdb7400b4975/map.xml,
1412027745352, FILE, null },pending,[(container_1411670948067_0009_02_000001)],443533288192637,DOWNLOADING}
> java.io.IOException: Rename cannot overwrite non empty destination directory /data/yarn/nm/filecache/27
> 	at org.apache.hadoop.fs.AbstractFileSystem.renameInternal(AbstractFileSystem.java:716)
> 	at org.apache.hadoop.fs.FilterFs.renameInternal(FilterFs.java:228)
> 	at org.apache.hadoop.fs.AbstractFileSystem.rename(AbstractFileSystem.java:659)
> 	at org.apache.hadoop.fs.FileContext.rename(FileContext.java:906)
> 	at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:366)
> 	at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:59)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message