hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (JIRA)" <j...@apache.org>
Subject [jira] Assigned: (MAPREDUCE-2049) JT and TT should prune invalid local dirs on startup
Date Wed, 08 Sep 2010 01:27:35 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Scott Chen reassigned MAPREDUCE-2049:
-------------------------------------

    Assignee: Zheng Shao

Zheng has made some change internally to fix this problem. I am assigning this to him.

> JT and TT should prune invalid local dirs on startup
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-2049
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2049
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, tasktracker
>    Affects Versions: 0.21.0, 0.22.0
>            Reporter: Eli Collins
>            Assignee: Zheng Shao
>            Priority: Minor
>             Fix For: 0.21.1, 0.22.0
>
>
> MRAsyncDiskService fails when it's given local dirs it can't create or write. Per MAPREDUCE-1213
this conflicts with the previous behavior. The TTs will no longer start on hosts where the
config has localdirs that can not be created (ie if some hosts don't have the same local mounts).
 
> The TT and JT should prune out invalid local dirs (ones that fail checkLocalDirs ie that
it can not create) before giving them to MRAsyncDiskService and using them otherwise.
> Also mapred-default.xml states that mapred.local.dir that do not exist are ignored, this
should be updated as directories that don't exist are created (if possible), they're only
ignored if they can not be created. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message