hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milind Bhandarkar (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-2115) Task cwds should be distributed across partitions
Date Sun, 28 Oct 2007 19:35:50 GMT
Task cwds should be distributed across partitions

                 Key: HADOOP-2115
                 URL: https://issues.apache.org/jira/browse/HADOOP-2115
             Project: Hadoop
          Issue Type: Improvement
          Components: mapred
    Affects Versions: 0.14.3
         Environment: All
            Reporter: Milind Bhandarkar
             Fix For: 0.16.0

Even when mapred.local.dir specifies a comma-separated list of partitions (typically one per
physical disk), all tasks of the same job have current working directories that belong to
only one partition. For side-effect tasks, that use local cwd as a scratch space, this overloads
a single disk while other disks may be idle. Idially, each task should get a cwd on different
partition. This is related to HADOOP-1991, but emphasizes performance impact.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message