Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3C0EC1085A for ; Sat, 31 Aug 2013 02:22:36 +0000 (UTC) Received: (qmail 86203 invoked by uid 500); 31 Aug 2013 02:22:31 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 85957 invoked by uid 500); 31 Aug 2013 02:22:31 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 85950 invoked by uid 99); 31 Aug 2013 02:22:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 31 Aug 2013 02:22:31 +0000 X-ASF-Spam-Status: No, hits=-2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_HI,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.91.2.13] (HELO smtp-outbound-2.vmware.com) (208.91.2.13) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 31 Aug 2013 02:22:27 +0000 Received: from sc9-mailhost2.vmware.com (sc9-mailhost2.vmware.com [10.113.161.72]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id 1DA8E28D80; Fri, 30 Aug 2013 19:22:07 -0700 (PDT) Received: from zcs-prod-mta-2.vmware.com (zcs-prod-mta-2.vmware.com [10.113.163.64]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id 119F3B04DB; Fri, 30 Aug 2013 19:22:07 -0700 (PDT) Received: from zcs-prod-mta-2.vmware.com (localhost.localdomain [127.0.0.1]) by zcs-prod-mta-2.vmware.com (Postfix) with ESMTP id 0293BE0049; Fri, 30 Aug 2013 19:22:07 -0700 (PDT) Received: from zcs-prod-mbox-41.vmware.com (lbv-sc9-t2prod2-int.vmware.com [10.113.160.246]) by zcs-prod-mta-2.vmware.com (Postfix) with ESMTP; Fri, 30 Aug 2013 19:22:06 -0700 (PDT) Date: Fri, 30 Aug 2013 19:22:06 -0700 (PDT) From: Jun Ping Du To: user@hadoop.apache.org Cc: baskar.duraikannu@outlook.com, rahul.rec.dgp@gmail.com, amuise@hortonworks.com Message-ID: <1281864.42056806.1377915726662.JavaMail.root@vmware.com> In-Reply-To: References: Subject: Re: Multidata center support MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_42056805_1565732486.1377915726661" X-Originating-IP: [10.113.62.222] X-Mailer: Zimbra 8.0.3_GA_5664 (ZimbraWebClient - SAF5.1 (Mac)/8.0.3_GA_5664) Thread-Topic: Multidata center support Thread-Index: Br+cMDLArDIrPEEBZvrblYVcB0U3ww== X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_42056805_1565732486.1377915726661 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, Although you can set datacenter layer on your network topology, it is never enabled in hadoop as lacking of replica placement and task scheduling support. There are some work to add layers other than rack and node under HADOOP-8848 but may not suit for your case. Agree with Adam that a cluster spanning multiple data centers seems not make sense even for DR case. Do you have other cases to do such a deployment? Thanks, Junping ----- Original Message ----- From: "Adam Muise" To: user@hadoop.apache.org Sent: Friday, August 30, 2013 6:26:54 PM Subject: Re: Multidata center support Nothing has changed. DR best practice is still one (or more) clusters per site and replication is handled via distributed copy or some variation of it. A cluster spanning multiple data centers is a poor idea right now. On Fri, Aug 30, 2013 at 12:35 AM, Rahul Bhattacharjee < rahul.rec.dgp@gmail.com > wrote: My take on this. Why hadoop has to know about data center thing. I think it can be installed across multiple data centers , however topology configuration would be required to tell which node belongs to which data center and switch for block placement. Thanks, Rahul On Fri, Aug 30, 2013 at 12:42 AM, Baskar Duraikannu < baskar.duraikannu@outlook.com > wrote:
We have a need to setup hadoop across data centers. Does hadoop support multi data center configuration? I searched through archives and have found that hadoop did not support multi data center configuration some time back. Just wanted to see whether situation has changed. Please help.
-- Adam Muise Solution Engineer Hortonworks amuise@hortonworks.com 416-417-4037 Hortonworks - Develops, Distributes and Supports Enterprise Apache Hadoop. Hortonworks Virtual Sandbox Hadoop: Disruptive Possibilities by Jeff Needham CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. ------=_Part_42056805_1565732486.1377915726661 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi,
    Alth= ough you can set datacenter layer on your network topology, it is neve= r enabled in hadoop as lacking of replica placement and task scheduling sup= port. There are some work to add layers other than rack and node under HADO= OP-8848 but may not suit for your case. Agree with Adam that a cl= uster spanning multiple data centers seems not make sense even for DR case.=  Do you have other cases to do such a deployment?

=
Thanks,

Junping

= From: "Adam Muise" <amuise@hortonworks.com>
To: user@ha= doop.apache.org
Sent: Friday, August 30, 2013 6:26:54 PM
Su= bject: Re: Multidata center support

= Nothing has changed. DR best practice is still one (or more) clusters per s= ite and replication is handled via distributed copy or some variation of it= . A cluster spanning multiple data centers is a poor idea right now.




On Fri, Aug 30, 2013 at 12:35 AM, Rahul Bhatta= charjee <rahul.rec.dgp@gmail.com> wrote:
My take on this.

Why hadoop has to know about data center thing. I think it can be installed= across multiple data centers , however topology configuration would be req= uired to tell which node belongs to which data center and switch for block = placement.

= Thanks,
Rahul


On Fr= i, Aug 30, 2013 at 12:42 AM, Baskar Duraikannu <baskar.duraika= nnu@outlook.com> wrote:
We have a need to setup hadoop across data centers. &= nbsp;Does hadoop support multi data center configuration? I searched throug= h archives and have found that hadoop did not support multi data center con= figuration some time back. Just wanted to see whether situation has changed= .

Please help.
=09=09 =09 =09=09




--
=

CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.

------=_Part_42056805_1565732486.1377915726661--