hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Johan Oskarson (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-297) When selecting node to put new block on, give priority to those with more free space/less blocks
Date Tue, 13 Jun 2006 16:24:29 GMT
When selecting node to put new block on, give priority to those with more free space/less blocks
------------------------------------------------------------------------------------------------

         Key: HADOOP-297
         URL: http://issues.apache.org/jira/browse/HADOOP-297
     Project: Hadoop
        Type: Improvement

  Components: dfs  
    Versions: 0.3.2    
    Reporter: Johan Oskarson
    Priority: Minor


As mentioned in previous bug report:
We're running a smallish cluster with very different machines, some with only 60 gb harddrives
This creates a problem when inserting files into the dfs, these machines run out of space
quickly while some have plenty of space free.

So instead of just shuffling the nodes, I've created a quick patch that first sorts the target
nodes by (freespace / blocks).
It then randomizes the position of the first third of the nodes (so we don't put all the blocks
in the file on the same machine)

I'll let you guys figure out how to improve this.
/Johan

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message