hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-7551) LocalDirAllocator should incorporate LocalStorage
Date Wed, 17 Aug 2011 23:18:27 GMT
LocalDirAllocator should incorporate LocalStorage
-------------------------------------------------

                 Key: HADOOP-7551
                 URL: https://issues.apache.org/jira/browse/HADOOP-7551
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs
    Affects Versions: 0.20.204.0
            Reporter: Eli Collins


The o.a.h.fs.LocalDirAllocator is not aware of o.a.h.m.t.LocalStorage (introduced in MAPREDUCE-2413)
- it always considers the configured local dirs, not just the ones that happen to be good.
Therefore if there's a disk failure then *every* call to get a local path will result in LocalDirAllocator#confChanged
doing a disk check of *all* the configured local dirs. It seems like LocalStorage should be
a private class to LocalAllocator so that all users of LocalDirAllocator benefit from the
disk failure handling and all the various users of LocalDirAllocator don't have to be modified
to handle disk failures. Note that LocalDirAllocator already handles faulty directories.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message