hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-692) Rack-aware Replica Placement
Date Tue, 14 Nov 2006 20:20:38 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-692?page=comments#action_12449767 ] 
Doug Cutting commented on HADOOP-692:

> Should we consider a mobile network [ .. . ]

I think it's easy to construct interfaces that permit this, but I don't think we should attempt
to support this in at least the initial implementation.

> it can not express the case when nodes are connected by a switch

How so?  It should be easy to implement this so that nodes connected by a switch are always
closer to one another than to any other nodes, no?

> Users may specify its replica placement policy

I see that as a subsequent feature.  I think in the first version simply placing things at
progressive distances until the replications are all placed will satisfy the majority of applications.

> Rack-aware Replica Placement
> ----------------------------
>                 Key: HADOOP-692
>                 URL: http://issues.apache.org/jira/browse/HADOOP-692
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.8.0
>            Reporter: Hairong Kuang
>         Assigned To: Hairong Kuang
>             Fix For: 0.9.0
> This issue assumes that HDFS runs on a cluster of computers that spread across many racks.
Communication between two nodes on different racks needs to go through switches. Bandwidth
in/out of a rack may be less than the total bandwidth of machines in the rack. The purpose
of rack-aware replica placement is to improve data reliability, availability, and network
bandwidth utilization. The basic idea is that each data node determines to which rack it belongs
at the startup time and notifies the name node of the rack id upon registration. The name
node maintains a rackid-to-datanode map and tries to place replicas across racks.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message