hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-74) hash blocks into dfs.data.dirs
Date Fri, 10 Mar 2006 22:33:54 GMT
hash blocks into dfs.data.dirs

         Key: HADOOP-74
         URL: http://issues.apache.org/jira/browse/HADOOP-74
     Project: Hadoop
        Type: Improvement
  Components: dfs  
    Versions: 0.1    
 Environment: large clusters
    Reporter: Doug Cutting
 Assigned to: Konstantin Shvachko 
     Fix For: 0.1

When dfs.data.dir has multiple values, we currently start a DataNode for each (all in the
same JVM).  Instead we should run a single DataNode that stores block files into the different
directories.  This will reduce the number of connections to the namenode.  We cannot hash
because different devices might be different amounts full.  So the datanode will need to keep
a table mapping from block id to file location, and add new blocks to less full devices.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message