Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9FC7717F90 for ; Wed, 1 Oct 2014 16:01:25 +0000 (UTC) Received: (qmail 54012 invoked by uid 500); 1 Oct 2014 16:01:25 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 53938 invoked by uid 500); 1 Oct 2014 16:01:25 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 53929 invoked by uid 99); 1 Oct 2014 16:01:25 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2014 16:01:25 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 1F29C925119; Wed, 1 Oct 2014 16:01:25 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: arp@apache.org To: common-commits@hadoop.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: git commit: HDFS-6581. Update CHANGES.txt in preparation for trunk merge Date: Wed, 1 Oct 2014 16:01:25 +0000 (UTC) Repository: hadoop Updated Branches: refs/heads/HDFS-6581 e431d054c -> 04b08431a HDFS-6581. Update CHANGES.txt in preparation for trunk merge Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/04b08431 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/04b08431 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/04b08431 Branch: refs/heads/HDFS-6581 Commit: 04b08431a3446300f4715cf135f0e60f85e5bf5a Parents: e431d05 Author: arp Authored: Wed Oct 1 09:01:08 2014 -0700 Committer: arp Committed: Wed Oct 1 09:01:08 2014 -0700 ---------------------------------------------------------------------- .../hadoop-hdfs/CHANGES-HDFS-6581.txt | 96 ------------------ hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 100 +++++++++++++++++++ 2 files changed, 100 insertions(+), 96 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/04b08431/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt deleted file mode 100644 index b41e133..0000000 --- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt +++ /dev/null @@ -1,96 +0,0 @@ - BREAKDOWN OF HDFS-6581 SUBTASKS AND RELATED JIRAS - - HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) - - HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) - - HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and - edit logs. (Arpit Agarwal) - - HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol. - (Arpit Agarwal) - - HDFS-6925. DataNode should attempt to place replicas on transient storage - first if lazyPersist flag is received. (Arpit Agarwal) - - HDFS-6926. DN support for saving replicas to persistent storage and - evicting in-memory replicas. (Arpit Agarwal) - - HDFS-6927. Initial unit tests for lazy persist files. (Arpit Agarwal) - - HDFS-6929. NN periodically unlinks lazy persist files with missing - replicas from namespace. (Arpit Agarwal) - - HDFS-6928. 'hdfs put' command should accept lazyPersist flag for testing. - (Arpit Agarwal) - - HDFS-6960. Bugfix in LazyWriter, fix test case and some refactoring. - (Arpit Agarwal) - - HDFS-6931. Move lazily persisted replicas to finalized directory on DN - startup. (Arpit Agarwal) - - HDFS-6950. Add Additional unit tests for HDFS-6581. (Xiaoyu Yao via - Arpit Agarwal) - - HDFS-6930. Improve replica eviction from RAM disk. (Arpit Agarwal) - - HDFS-6977. Delete all copies when a block is deleted from the block space. - (Arpit Agarwal) - - HDFS-6991. Notify NN of evicted block before deleting it from RAM disk. - (Arpit Agarwal) - - HDFS-6978. Directory scanner should correctly reconcile blocks on RAM - disk. (Arpit Agarwal) - - HDFS-7066. LazyWriter#evictBlocks misses a null check for replicaState. - (Xiaoyu Yao via Arpit Agarwal) - - HDFS-7064. Fix unit test failures in HDFS-6581 branch. (Xiaoyu Yao via - Arpit Agarwal) - - HDFS-6581. Few more unit test fixes for HDFS-6581. (Arpit Agarwal) - - HDFS-7080. Fix finalize and upgrade unit test failures. (Arpit Agarwal) - - HDFS-7084. FsDatasetImpl#copyBlockFiles debug log can be improved. - (Xiaoyu Yao via Arpit Agarwal) - - HDFS-7091. Add forwarding constructor for INodeFile for existing callers. - (Arpit Agarwal) - - HDFS-7100. Make eviction scheme pluggable. (Arpit Agarwal) - - HDFS-7108. Fix unit test failures in SimulatedFsDataset. (Arpit Agarwal) - - HDFS-7071. Updated editsStored and editsStored.xml to bump layout - version and add LazyPersist flag. (Xiaoyu Yao and Arpit Agarwal via - Arpit Agarwal) - - HDFS-6990. Add unit test for evict/delete RAM_DISK block with open - handle. (Xiaoyu Yao via Arpit Agarwal) - - HDFS-7143. Fix findbugs warnings in HDFS-6581 branch. (szetszwo via - Arpit Agarwal) - - HDFS-6932. Balancer and Mover tools should ignore replicas on RAM_DISK. - (Xiaoyu Yao via Arpit Agarwal) - - HDFS-7144. Fix findbugs warnings in RamDiskReplicaTracker. (szetszwo via - Arpit Agarwal) - - HDFS-7155. Bugfix in createLocatedFileStatus caused by bad merge. - (Arpit Agarwal) - - HDFS-7153. Add storagePolicy to NN edit log during file creation. - (Arpit Agarwal) - - HDFS-7159. Use block storage policy to set lazy persist preference. - (Arpit Agarwal) - - HDFS-7129. Metrics to track usage of memory for writes. (Xiaoyu Yao - via Arpit Agarwal) - - HDFS-7171. Fix Jenkins failures in HDFS-6581 branch. (Arpit Agarwal) - http://git-wip-us.apache.org/repos/asf/hadoop/blob/04b08431/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt index 4438d35..f66b927 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt +++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt @@ -18,6 +18,9 @@ Trunk (Unreleased) HDFS-3125. Add JournalService to enable Journal Daemon. (suresh) + HDFS-6581. Support writing to a single replica in RAM. (See breakdown of + tasks below for features and contributors). + IMPROVEMENTS HDFS-4665. Move TestNetworkTopologyWithNodeGroup to common. @@ -258,6 +261,103 @@ Trunk (Unreleased) HDFS-6981. Fix DN upgrade with layout version change. (Arpit Agarwal) + BREAKDOWN OF HDFS-6581 SUBTASKS AND RELATED JIRAS + + HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) + + HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) + + HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and + edit logs. (Arpit Agarwal) + + HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol. + (Arpit Agarwal) + + HDFS-6925. DataNode should attempt to place replicas on transient storage + first if lazyPersist flag is received. (Arpit Agarwal) + + HDFS-6926. DN support for saving replicas to persistent storage and + evicting in-memory replicas. (Arpit Agarwal) + + HDFS-6927. Initial unit tests for lazy persist files. (Arpit Agarwal) + + HDFS-6929. NN periodically unlinks lazy persist files with missing + replicas from namespace. (Arpit Agarwal) + + HDFS-6928. 'hdfs put' command should accept lazyPersist flag for testing. + (Arpit Agarwal) + + HDFS-6960. Bugfix in LazyWriter, fix test case and some refactoring. + (Arpit Agarwal) + + HDFS-6931. Move lazily persisted replicas to finalized directory on DN + startup. (Arpit Agarwal) + + HDFS-6950. Add Additional unit tests for HDFS-6581. (Xiaoyu Yao via + Arpit Agarwal) + + HDFS-6930. Improve replica eviction from RAM disk. (Arpit Agarwal) + + HDFS-6977. Delete all copies when a block is deleted from the block space. + (Arpit Agarwal) + + HDFS-6991. Notify NN of evicted block before deleting it from RAM disk. + (Arpit Agarwal) + + HDFS-6978. Directory scanner should correctly reconcile blocks on RAM + disk. (Arpit Agarwal) + + HDFS-7066. LazyWriter#evictBlocks misses a null check for replicaState. + (Xiaoyu Yao via Arpit Agarwal) + + HDFS-7064. Fix unit test failures in HDFS-6581 branch. (Xiaoyu Yao via + Arpit Agarwal) + + HDFS-6581. Few more unit test fixes for HDFS-6581. (Arpit Agarwal) + + HDFS-7080. Fix finalize and upgrade unit test failures. (Arpit Agarwal) + + HDFS-7084. FsDatasetImpl#copyBlockFiles debug log can be improved. + (Xiaoyu Yao via Arpit Agarwal) + + HDFS-7091. Add forwarding constructor for INodeFile for existing callers. + (Arpit Agarwal) + + HDFS-7100. Make eviction scheme pluggable. (Arpit Agarwal) + + HDFS-7108. Fix unit test failures in SimulatedFsDataset. (Arpit Agarwal) + + HDFS-7071. Updated editsStored and editsStored.xml to bump layout + version and add LazyPersist flag. (Xiaoyu Yao and Arpit Agarwal via + Arpit Agarwal) + + HDFS-6990. Add unit test for evict/delete RAM_DISK block with open + handle. (Xiaoyu Yao via Arpit Agarwal) + + HDFS-7143. Fix findbugs warnings in HDFS-6581 branch. (szetszwo via + Arpit Agarwal) + + HDFS-6932. Balancer and Mover tools should ignore replicas on RAM_DISK. + (Xiaoyu Yao via Arpit Agarwal) + + HDFS-7144. Fix findbugs warnings in RamDiskReplicaTracker. (szetszwo via + Arpit Agarwal) + + HDFS-7155. Bugfix in createLocatedFileStatus caused by bad merge. + (Arpit Agarwal) + + HDFS-7153. Add storagePolicy to NN edit log during file creation. + (Arpit Agarwal) + + HDFS-7159. Use block storage policy to set lazy persist preference. + (Arpit Agarwal) + + HDFS-7129. Metrics to track usage of memory for writes. (Xiaoyu Yao + via Arpit Agarwal) + + HDFS-7171. Fix Jenkins failures in HDFS-6581 branch. (Arpit Agarwal) + + Release 2.7.0 - UNRELEASED INCOMPATIBLE CHANGES