Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5878210CDD for ; Mon, 21 Dec 2015 12:48:31 +0000 (UTC) Received: (qmail 61472 invoked by uid 500); 21 Dec 2015 12:48:31 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 61405 invoked by uid 500); 21 Dec 2015 12:48:30 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 61396 invoked by uid 99); 21 Dec 2015 12:48:30 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Dec 2015 12:48:30 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 80124E04AF; Mon, 21 Dec 2015 12:48:30 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: aajisaka@apache.org To: common-commits@hadoop.apache.org Message-Id: <0de3af41cfc4484f982f93f9fc74870e@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: hadoop git commit: HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke. Date: Mon, 21 Dec 2015 12:48:30 +0000 (UTC) Repository: hadoop Updated Branches: refs/heads/branch-2.7 5ffad1551 -> e0e5a0fa8 HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke. (cherry picked from commit 049ccf71ce4457296974f12f5159540f8fcf1ede) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/e0e5a0fa Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/e0e5a0fa Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/e0e5a0fa Branch: refs/heads/branch-2.7 Commit: e0e5a0fa828a975db0b6d1b02fb460d412edc2bb Parents: 5ffad15 Author: Haohui Mai Authored: Sun Nov 22 16:10:27 2015 -0800 Committer: Akira Ajisaka Committed: Mon Dec 21 21:44:46 2015 +0900 ---------------------------------------------------------------------- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 3 +++ hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/e0e5a0fa/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt index 2dad550b..92e0ef0 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt +++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt @@ -42,6 +42,9 @@ Release 2.7.3 - UNRELEASED HDFS-9505. HDFS Architecture documentation needs to be refreshed. (Masatake Iwasaki via aajisaka) + HDFS-8914. Document HA support in the HDFS HdfsDesign.md. + (Lars Francke via wheat9) + Release 2.7.2 - UNRELEASED INCOMPATIBLE CHANGES http://git-wip-us.apache.org/repos/asf/hadoop/blob/e0e5a0fa/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md index 786db42..482b007 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md @@ -193,7 +193,7 @@ It is possible that a block of data fetched from a DataNode arrives corrupted. T The FsImage and the EditLog are central data structures of HDFS. A corruption of these files can cause the HDFS instance to be non-functional. For this reason, the NameNode can be configured to support maintaining multiple copies of the FsImage and EditLog. Any update to either the FsImage or EditLog causes each of the FsImages and EditLogs to get updated synchronously. This synchronous updating of multiple copies of the FsImage and EditLog may degrade the rate of namespace transactions per second that a NameNode can support. However, this degradation is acceptable because even though HDFS applications are very data intensive in nature, they are not metadata intensive. When a NameNode restarts, it selects the latest consistent FsImage and EditLog to use. -The NameNode machine is a single point of failure for an HDFS cluster. If the NameNode machine fails, manual intervention is necessary. Currently, automatic restart and failover of the NameNode software to another machine is not supported. +Another option to increase resilience against failures is to enable High Availability using multiple NameNodes either with a [shared storage on NFS](./HDFSHighAvailabilityWithNFS.html) or using a [distributed edit log](./HDFSHighAvailabilityWithQJM.html) (called Journal). The latter is the recommended approach. ### Snapshots