hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1288) DistributedCache localizes only once per cache URI
Date Fri, 11 Dec 2009 17:05:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789390#action_12789390
] 

Allen Wittenauer commented on MAPREDUCE-1288:
---------------------------------------------

What happens in the case that the archive file changes in flight.  For example, I submit a
job using that archive.  While my job is running, I notice a bug, remove the old cache file,
push a new one to hdfs, and then launch a new invocation of my job.  Would the new job get
the old cache file because the old job is still running?

> DistributedCache localizes only once per cache URI
> --------------------------------------------------
>
>                 Key: MAPREDUCE-1288
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1288
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.21.0
>
>
> As part of the file localization the distributed cache localizer creates a copy of the
file in the corresponding user's private directory. The localization in DistributedCache assumes
the key as the URI of the cachefile and if it already exists in the map, the localization
is not done again. This means that another user cannot access the same distributed cache file.
We should change the key to include the username so that localization is done for every user.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message