hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sameer Paranjpye (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-64) DataNode should be capable of managing multiple volumes
Date Thu, 23 Mar 2006 01:28:26 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-64?page=comments#action_12371504 ] 

Sameer Paranjpye commented on HADOOP-64:
----------------------------------------

Heartbeat overhead aside, a second motivator for this change is the simplification of block
placement logic. The namenode wouldn't have to track the datanode-machine mapping to ensure
that replicas of a block don't end up on the same box. I don't know if this is currently done,
but if not it could be a problem. Also, if block placement decisions are to be made based
on some measure of load (number of blocks in being read/written, CPU usage etc.) the load
needs to be tracked at a machine level. It's simpler to note have the namenode compute the
aggregate load per machine.

> DataNode should be capable of managing multiple volumes
> -------------------------------------------------------
>
>          Key: HADOOP-64
>          URL: http://issues.apache.org/jira/browse/HADOOP-64
>      Project: Hadoop
>         Type: Improvement
>   Components: dfs
>     Versions: 0.2
>     Reporter: Sameer Paranjpye
>     Assignee: Konstantin Shvachko
>     Priority: Minor
>      Fix For: 0.2

>
> The dfs Datanode can only store data on a single filesystem volume. When a node runs
its disks JBOD this means running a Datanode per disk on the machine. While the scheme works
reasonably well on small clusters, on larger installations (several 100 nodes) it implies
a very large number of Datanodes with associated management overhead in the Namenode.
> The Datanod should be enhanced to be able to handle multiple volumes on a single machine.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message