hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Omkar Vinit Joshi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1036) Distributed Cache gives inconsistent result if cache files get deleted from task tracker
Date Wed, 07 Aug 2013 19:51:48 GMT

    [ https://issues.apache.org/jira/browse/YARN-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732653#comment-13732653

Omkar Vinit Joshi commented on YARN-1036:

[~raviprak] Let me try to explain again.
* Resource R1 is not in the cache. so on (REQUEST) new resource was created in (DOWNLOADING)
state and download started.
* Once download is finished; we get (LOCALIZED) (here we don't want to check whether file
is present on disc or not.. do we have to? if yes then probably we may have to update trunk
also as I separated the logic in one of the jiras.) and we change the resource state to (LOCALIZED)
and notify all waiting containers. [File is present on disc].
* Now file got deleted.
* Another container requests the file (REQUEST).. now we should definitely check if the file
is already present on disc or not. If not remove it from local cache (DataStructure-- by delete
the file I wanted to say from local cache not from disc as it is already gone from there).
Let me know your thoughts.
> Distributed Cache gives inconsistent result if cache files get deleted from task tracker

> -----------------------------------------------------------------------------------------
>                 Key: YARN-1036
>                 URL: https://issues.apache.org/jira/browse/YARN-1036
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 0.23.9
>            Reporter: Ravi Prakash
>            Assignee: Ravi Prakash
>         Attachments: YARN-1036.branch-0.23.patch
> This is a JIRA to backport MAPREDUCE-4342. I had to open a new JIRA because that one
had been closed. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message