hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3011) TT should remove bad local dirs from conf to prevent constant disk checking
Date Thu, 13 Oct 2011 06:45:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126387#comment-13126387
] 

Ravi Gummadi commented on MAPREDUCE-3011:
-----------------------------------------

LocalDirAllocator#AllocatorPerContext#confChanged() is actually updating the savedLocalDirs
everytime conf is changed. So every call to confChanged() is not resulting into disk checks
(except once per configuration change i.e. once per every bad disk). Right ?
                
> TT should remove bad local dirs from conf to prevent constant disk checking
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3011
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3011
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: tasktracker
>    Affects Versions: 0.20.204.0
>            Reporter: Eli Collins
>             Fix For: 0.20.205.0
>
>
> Per HADOOP-7551 the TT does not remove bad mapred.local.dirs from the conf so after a
single disk failure *every* call to get a local path for reading or writing results in a disk
check of *all* configured local dirs. After detecting that a local dir is bad we should remove
it from the conf so that we don't repeatedly perform this expensive operation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message