hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-1799) Enhance LocalDirAllocator in NM to consider DiskMaxUtilization cutoff
Date Fri, 07 Mar 2014 05:00:44 GMT
Sunil G created YARN-1799:
-----------------------------

             Summary: Enhance LocalDirAllocator in NM to consider DiskMaxUtilization cutoff
                 Key: YARN-1799
                 URL: https://issues.apache.org/jira/browse/YARN-1799
             Project: Hadoop YARN
          Issue Type: Sub-task
          Components: nodemanager
    Affects Versions: 2.3.0
            Reporter: Sunil G


LocalDirAllocator provides paths for all tasks for its local write.
This considers the good list of directories which are selected by the HealthCheck mechamnism
in LocalDirsHandlerService

getLocalPathForWrite() considers whether input demand "size" can meet the "capacity" in "lastAccessed"
directory.
If more tasks asks for path from LocalDirAllocator, then it is possible that the allocation
is done based on the current disk availability at that given time.
But this path would have earlier given to some other tasks to write and they may be sequentially
doing writing.

It is better to check for an upper cutoff for disk availability



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message