hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhong Wang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1225) TT successfully localizes a task even though the corresponding cache-file has already changed on DFS.
Date Tue, 20 Apr 2010 14:56:51 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhong Wang updated MAPREDUCE-1225:
----------------------------------

    Attachment: MAPREDUCE-1225.patch

Updated patch from Amareshwari. Patch does the following:
1. Refactor isExistsAndFresh(). Move the first check into checkStampSinceJobStarted().
2. Remove the fileStatus argument from localizeCache(), since this arg is never used.
3. Add a test case in testFreshness() to test whether changing stamp of uninited cache could
fail a job.
4. Modify calls of getLocalCache in TestTrackerDistributedCacheManager to use the right timestamp.

Thanks a lot!

> TT successfully localizes a task even though the corresponding cache-file has already
changed on DFS.
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1225
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1225
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0, 0.22.0
>            Reporter: Vinod K V
>             Fix For: 0.21.0
>
>         Attachments: MAPREDUCE-1225.patch, MAPREDUCE-1225.patch, MAPREDUCE-1225.patch
>
>
> This happens with the first task of a job being localized on this TT. TT doesn't check
if the file on DFS is fresh according to the timestamps set in job-conf during submission.
After the first task incorrectly gets localized, all further tasks fail on this TT as expected.
> Found this issue while trying to improve test-case for MAPREUCE-913.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message