Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <1850528515.1260599778092.JavaMail.jira@brutus>
Date: Sat, 12 Dec 2009 06:36:18 +0000 (UTC)
From: "Devaraj Das (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-1288) DistributedCache localizes only
 once per cache URI
In-Reply-To: <970147752.1260484338104.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789696#action_12789696 ] 

Devaraj Das commented on MAPREDUCE-1288:
----------------------------------------

If i am reading the code right, the tasks of the new job would fail on those nodes that localized the old archive and still has a copy of that (the TaskTracker would detect the archive has changed and assuming that the change happened while the new job was running would fail the tasks). This will continue until the archive is purged from the cache and re-localized.

> DistributedCache localizes only once per cache URI
> --------------------------------------------------
>
>                 Key: MAPREDUCE-1288
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1288
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.21.0
>
>
> As part of the file localization the distributed cache localizer creates a copy of the file in the corresponding user's private directory. The localization in DistributedCache assumes the key as the URI of the cachefile and if it already exists in the map, the localization is not done again. This means that another user cannot access the same distributed cache file. We should change the key to include the username so that localization is done for every user.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.