hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evert Lammerts (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-2636) Scheduling over disks horizontally
Date Fri, 01 Jul 2011 08:30:29 GMT
Scheduling over disks horizontally

                 Key: MAPREDUCE-2636
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2636
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: job submission
            Reporter: Evert Lammerts
            Priority: Minor

Based on this message: http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201106.mbox/browser

The JT schedules tasks on nodes based on metadata it gets from the NN. The namenode does not
know on which disk a block resides. It might happen that on a node running 4 tasks, all read
from the same disk. This can affect performance.

An optimization might be to schedule horizontally over disks instead of nodes. Any ideas?

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message