hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-3011) TT should remove bad local dirs from conf to prevent constant disk checking
Date Thu, 03 Nov 2011 20:11:32 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Eli Collins resolved MAPREDUCE-3011.

    Resolution: Not A Problem

You're right, I missed that we're always updating the conf the job is launched with the latest
known good dirs in TT#launchTaskForJob.  Thanks for the explanation.  I verified we weren't
missing other locations by failing a local dir, logging in the confChanged case and verifying
that we only notice the change once.
> TT should remove bad local dirs from conf to prevent constant disk checking
> ---------------------------------------------------------------------------
>                 Key: MAPREDUCE-3011
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3011
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: tasktracker
>    Affects Versions:
>            Reporter: Eli Collins
> Per HADOOP-7551 the TT does not remove bad mapred.local.dirs from the conf so after a
single disk failure *every* call to get a local path for reading or writing results in a disk
check of *all* configured local dirs. After detecting that a local dir is bad we should remove
it from the conf so that we don't repeatedly perform this expensive operation.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message