Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2352F110E9 for ; Mon, 9 Jun 2014 09:21:07 +0000 (UTC) Received: (qmail 73219 invoked by uid 500); 9 Jun 2014 09:21:06 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 73168 invoked by uid 500); 9 Jun 2014 09:21:06 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 73157 invoked by uid 99); 9 Jun 2014 09:21:06 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Jun 2014 09:21:06 +0000 Date: Mon, 9 Jun 2014 09:21:06 +0000 (UTC) From: "Dian Fu (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-5442) Zero loss HDFS data replication for multiple datacenters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated HDFS-5442: -------------------------- Attachment: Disaster Recovery Solution for Hadoop.pdf Updated the design doc, add some detailed implementation. > Zero loss HDFS data replication for multiple datacenters > -------------------------------------------------------- > > Key: HDFS-5442 > URL: https://issues.apache.org/jira/browse/HDFS-5442 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Avik Dey > Assignee: Dian Fu > Attachments: Disaster Recovery Solution for Hadoop.pdf, Disaster Recovery Solution for Hadoop.pdf, Disaster Recovery Solution for Hadoop.pdf > > > Hadoop is architected to operate efficiently at scale for normal hardware failures within a datacenter. Hadoop is not designed today to handle datacenter failures. Although HDFS is not designed for nor deployed in configurations spanning multiple datacenters, replicating data from one location to another is common practice for disaster recovery and global service availability. There are current solutions available for batch replication using data copy/export tools. However, while providing some backup capability for HDFS data, they do not provide the capability to recover all your HDFS data from a datacenter failure and be up and running again with a fully operational Hadoop cluster in another datacenter in a matter of minutes. For disaster recovery from a datacenter failure, we should provide a fully distributed, zero data loss, low latency, high throughput and secure HDFS data replication solution for multiple datacenter setup. > Design and code for Phase-1 to follow soon. -- This message was sent by Atlassian JIRA (v6.2#6252)