hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Stretched HDFS cluster
Date Wed, 16 Sep 2009 07:15:01 GMT
Hi Gregory,

This is way outside the design parameters of HDFS. It may work, but you are
very likely to run into issues, and I don't think anyone would recommend
this as a solution. More reasonable would be a HDFS cluster spanning two
datacenters within the same metro area (1-2ms latency), but even that isn't
at all common.

For cross-site redundancy of HDFS, the usual solution is using distcp to
make periodic backups between separate clusters.


On Wed, Sep 16, 2009 at 12:09 AM, Touretsky, Gregory <
gregory.touretsky@intel.com> wrote:

> Hi,
>    Does anyone have an experience running HDFS cluster stretched over
> high-latency WAN connections?
> Any specific concerns/options/recommendations?
> I'm trying to setup the HDFS cluster with the nodes located in the US,
> Israel and India - considering it as a potential solution for cross-site
> data sharing...
> Regards,
> Gregory Touretsky
> Intel IT - Strategic Solutions and Architecture
> Systems Analyst
> gregory.touretsky AT intel.com
> (+) 972-4-865-6377, Fax: 04-865-5999
> iNET: 465-6377, M/S: IDC10-2.3
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message