Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 170BC18ED9 for ; Tue, 1 Mar 2016 21:30:48 +0000 (UTC) Received: (qmail 96978 invoked by uid 500); 1 Mar 2016 21:30:38 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 96904 invoked by uid 500); 1 Mar 2016 21:30:38 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 96891 invoked by uid 99); 1 Mar 2016 21:30:38 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Mar 2016 21:30:38 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id E3352E0007; Tue, 1 Mar 2016 21:30:37 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: cnauroth@apache.org To: common-commits@hadoop.apache.org Date: Tue, 01 Mar 2016 21:30:37 -0000 Message-Id: <2efe61d32ee0480bbd9163ae096a82bc@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/3] hadoop git commit: HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty. Repository: hadoop Updated Branches: refs/heads/branch-2 023c2d2e5 -> bce33b517 refs/heads/branch-2.8 c50f79d83 -> f563c5353 refs/heads/trunk 5c465df90 -> f98dff329 HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/f98dff32 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/f98dff32 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/f98dff32 Branch: refs/heads/trunk Commit: f98dff329b1f94c9f53022baf0209fc1a7aaf7c2 Parents: 5c465df Author: Chris Nauroth Authored: Tue Mar 1 13:18:42 2016 -0800 Committer: Chris Nauroth Committed: Tue Mar 1 13:18:42 2016 -0800 ---------------------------------------------------------------------- hadoop-common-project/hadoop-common/CHANGES.txt | 3 +++ hadoop-tools/hadoop-azure/src/site/markdown/index.md | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/f98dff32/hadoop-common-project/hadoop-common/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/CHANGES.txt b/hadoop-common-project/hadoop-common/CHANGES.txt index b3413cc..401df94 100644 --- a/hadoop-common-project/hadoop-common/CHANGES.txt +++ b/hadoop-common-project/hadoop-common/CHANGES.txt @@ -1218,6 +1218,9 @@ Release 2.8.0 - UNRELEASED HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases. (szetszwo) + HADOOP-12853. Change WASB documentation regarding page blob support. + (Madhumita Chakraborty via cnauroth) + BUG FIXES HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals http://git-wip-us.apache.org/repos/asf/hadoop/blob/f98dff32/hadoop-tools/hadoop-azure/src/site/markdown/index.md ---------------------------------------------------------------------- diff --git a/hadoop-tools/hadoop-azure/src/site/markdown/index.md b/hadoop-tools/hadoop-azure/src/site/markdown/index.md index d231f22..a4a7615 100644 --- a/hadoop-tools/hadoop-azure/src/site/markdown/index.md +++ b/hadoop-tools/hadoop-azure/src/site/markdown/index.md @@ -40,8 +40,8 @@ on the additional artifacts it requires, notably the * Present a hierarchical file system view by implementing the standard Hadoop [`FileSystem`](../api/org/apache/hadoop/fs/FileSystem.html) interface. * Supports configuration of multiple Azure Blob Storage accounts. -* Supports both page blobs (suitable for most use cases, such as MapReduce) and - block blobs (suitable for continuous write use cases, such as an HBase +* Supports both block blobs (suitable for most use cases, such as MapReduce) and + page blobs (suitable for continuous write use cases, such as an HBase write-ahead log). * Reference file system paths using URLs using the `wasb` scheme. * Also reference file system paths using URLs with the `wasbs` scheme for SSL @@ -168,9 +168,9 @@ can be written any number of times, whereas block blobs can only be appended to work for HBase logs, so page blob support was introduced to overcome this limitation. -Page blobs can be used for other purposes beyond just HBase log files though. Page blobs can be up to 1TB in size, larger than the maximum 200GB size for block blobs. +You should stick to block blobs for most usage, and page blobs are only tested in context of HBase write-ahead logs. In order to have the files you create be page blobs, you must set the configuration variable `fs.azure.page.blob.dir` to a comma-separated list of