hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Ping Du <...@vmware.com>
Subject Re: data center aware hadoop?
Date Sat, 22 Sep 2012 00:09:10 GMT
Hi Sujee,
   HDFS today doesn't consider too much on data center level reliability (although it is supposed
to extend to data center layer in topology but never honored in replica policemen/balancer/task
scheduling policy) and performance is part of concern to cross data center (assume cross-dc
bandwidth is lower than within data center). However, in future, I think we should deliver
a solution to enable data center level disaster recovery even performance is downgrade. My
several years experience in delivering enterprise software is: it is best to let customer
to make trade-off decision on performance and reliability, and engineering effort is to provide
options.
BTW, HDFS HA is a protection of key nodes from SPOF but not handle the whole data center shutdown.

Thanks,

Junping

----- Original Message -----
From: "Sujee Maniyam" <sujee@sujee.net>
To: "hdfs-dev" <hdfs-dev@hadoop.apache.org>
Sent: Tuesday, September 11, 2012 7:29:39 AM
Subject: data center aware hadoop?

HI devs
now that hfds HA is is a reality,  how about HDFS spanning multiple
data centers?  Are there any discussions / work going on in this area?

It could be a single cluster spanning multiple data centers or having
a 'standby cluster' in another data center.

curious, and thanks for your time!

regards
Sujee Maniyam
http://sujee.net

Mime
View raw message