Return-Path: X-Original-To: apmail-hadoop-hdfs-commits-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 323C29637 for ; Tue, 28 Feb 2012 02:22:12 +0000 (UTC) Received: (qmail 71379 invoked by uid 500); 28 Feb 2012 02:22:12 -0000 Delivered-To: apmail-hadoop-hdfs-commits-archive@hadoop.apache.org Received: (qmail 71351 invoked by uid 500); 28 Feb 2012 02:22:12 -0000 Mailing-List: contact hdfs-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-dev@hadoop.apache.org Delivered-To: mailing list hdfs-commits@hadoop.apache.org Received: (qmail 71342 invoked by uid 99); 28 Feb 2012 02:22:12 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Feb 2012 02:22:11 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Feb 2012 02:22:07 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 757642388A32; Tue, 28 Feb 2012 02:21:46 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1294445 - in /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs: ./ CHANGES.txt src/main/java/ src/main/native/ src/main/webapps/datanode/ src/main/webapps/hdfs/ src/main/webapps/secondary/ src/test/hdfs/ Date: Tue, 28 Feb 2012 02:21:46 -0000 To: hdfs-commits@hadoop.apache.org From: atm@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20120228022146.757642388A32@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: atm Date: Tue Feb 28 02:21:42 2012 New Revision: 1294445 URL: http://svn.apache.org/viewvc?rev=1294445&view=rev Log: Merge trunk into HA branch. Modified: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/ (props changed) hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/ (props changed) hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/native/ (props changed) hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/ (props changed) hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/ (props changed) hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/ (props changed) hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs/ (props changed) Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Feb 28 02:21:42 2012 @@ -1,4 +1,4 @@ -/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs:1152502-1294248 +/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs:1152502-1294443 /hadoop/core/branches/branch-0.19/hdfs:713112 /hadoop/hdfs/branches/HDFS-1052:987665-1095512 /hadoop/hdfs/branches/HDFS-265:796829-820463 Modified: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt?rev=1294445&r1=1294444&r2=1294445&view=diff ============================================================================== --- hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt (original) +++ hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Tue Feb 28 02:21:42 2012 @@ -3,93 +3,24 @@ Hadoop HDFS Change Log Trunk (unreleased changes) INCOMPATIBLE CHANGES - - HDFS-2676. Remove Avro RPC. (suresh) NEW FEATURES - - HDFS-395. DFS Scalability: Incremental block reports. (Tomasz Nykiel - via hairong) - - HDFS-2517. Add protobuf service for JounralProtocol. (suresh) - - HDFS-2518. Add protobuf service for NamenodeProtocol. (suresh) - - HDFS-2520. Add protobuf service for InterDatanodeProtocol. (suresh) - - HDFS-2519. Add protobuf service for DatanodeProtocol. (suresh) - - HDFS-2581. Implement protobuf service for JournalProtocol. (suresh) - - HDFS-2618. Implement protobuf service for NamenodeProtocol. (suresh) - - HDFS-2629. Implement protobuf service for InterDatanodeProtocol. (suresh) - - HDFS-2636. Implement protobuf service for ClientDatanodeProtocol. (suresh) HDFS-2430. The number of failed or low-resource volumes the NN can tolerate should be configurable. (atm) - HDFS-2642. Protobuf translators for DatanodeProtocol. (jitendra) - - HDFS-2647. Used protobuf based RPC for InterDatanodeProtocol, - ClientDatanodeProtocol, JournalProtocol, NamenodeProtocol. (suresh) - - HDFS-2666. Fix TestBackupNode failure. (suresh) - HDFS-234. Integration with BookKeeper logging system. (Ivan Kelly via jitendra) - HDFS-2663. Optional protobuf parameters are not handled correctly. - (suresh) - - HDFS-2661. Enable protobuf RPC for DatanodeProtocol. (jitendra) - - HDFS-2697. Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings - protocol to protocol buffers. (jitendra) - - HDFS-2880. Protobuf changes in DatanodeProtocol to add multiple storages. - (suresh) - - HDFS-2899. Service protocol changes in DatanodeProtocol to add multiple - storages. (suresh) - IMPROVEMENTS - HADOOP-7524 Change RPC to allow multiple protocols including multuple - versions of the same protocol (Sanjay Radia) - HDFS-1620. Rename HdfsConstants -> HdfsServerConstants, FSConstants -> HdfsConstants. (Harsh J Chouraria via atm) - HDFS-2197. Refactor RPC call implementations out of NameNode class (todd) - - HDFS-2018. Move all journal stream management code into one place. - (Ivan Kelly via jitendra) - - HDFS-2223. Untangle depencencies between NN components (todd) - HDFS-2337. DFSClient shouldn't keep multiple RPC proxy references (atm) - - HDFS-2351 Change Namenode and Datanode to register each of their protocols - seperately. (Sanjay Radia) + HDFS-2197. Refactor RPC call implementations out of NameNode class (todd) HDFS-2158. Add JournalSet to manage the set of journals. (jitendra) - HDFS-2459. Separate datatypes for JournalProtocol. (suresh) - - HDFS-2480. Separate datatypes for NamenodeProtocol. (suresh) - - HDFS-2181 Separate HDFS Client wire protocol data types (sanjay) - - HDFS-2489. Move Finalize and Register to separate file out of - DatanodeCommand.java. (suresh) - - HDFS-2488. Separate datatypes for InterDatanodeProtocol. (suresh) - - HDFS-2496. Separate datatypes for DatanodeProtocol. (suresh) - - HDFS-2479 HDFS Client Data Types in Protocol Buffers (sanjay) - HDFS-2334. Add Closeable to JournalManager. (Ivan Kelly via jitendra) HDFS-2572. Remove unnecessary double-check in DN#getHostName. (harsh) @@ -102,30 +33,12 @@ Trunk (unreleased changes) HDFS-2857. Cleanup BlockInfo class. (suresh) - HADOOP-7862 Hdfs changes to work with HADOOP 7862: - Move the support for multiple protocols to lower layer so that Writable, - PB and Avro can all use it (Sanjay) - HDFS-1580. Add interface for generic Write Ahead Logging mechanisms. (Ivan Kelly via jitendra) - HDFS-2597 ClientNameNodeProtocol in Protocol Buffers (sanjay) - - HDFS-2651 ClientNameNodeProtocol Translators for Protocol Buffers (sanjay) - - HDFS-2650. Replace @inheritDoc with @Override. (Hari Mankude via suresh) - - HDFS-2669. Enable protobuf rpc for ClientNamenodeProtocol. (Sanjay Radia) - - HDFS-2801. Provide a method in client side translators to check for a - methods supported in underlying protocol. (jitendra) - HDFS-208. name node should warn if only one dir is listed in dfs.name.dir. (Uma Maheswara Rao G via eli) - HDS-2895. Remove Writable wire protocol types and translators to - complete transition to protocol buffers. (suresh) - HDFS-2786. Fix host-based token incompatibilities in DFSUtil. (Kihwal Lee via jitendra) @@ -148,6 +61,7 @@ Trunk (unreleased changes) (suresh) OPTIMIZATIONS + HDFS-2477. Optimize computing the diff between a block report and the namenode state. (Tomasz Nykiel via hairong) @@ -158,6 +72,7 @@ Trunk (unreleased changes) over-replicated, and invalidated blocks. (Tomasz Nykiel via todd) BUG FIXES + HDFS-2299. TestOfflineEditsViewer is failing on trunk. (Uma Maheswara Rao G via atm) HDFS-2310. TestBackupNode fails since HADOOP-7524 went in. @@ -180,10 +95,118 @@ Trunk (unreleased changes) HDFS-2188. Make FSEditLog create its journals from a list of URIs rather than NNStorage. (Ivan Kelly via jitendra) - HDFS-2481 Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol. + HDFS-1765. Block Replication should respect under-replication + block priority. (Uma Maheswara Rao G via eli) + + HDFS-2765. TestNameEditsConfigs is incorrectly swallowing IOE. (atm) + + HDFS-2776. Missing interface annotation on JournalSet. + (Brandon Li via jitendra) + + HDFS-2759. Pre-allocate HDFS edit log files after writing version number. + (atm) + + HDFS-2908. Add apache license header for StorageReport.java. (Brandon Li + via jitendra) + +Release 0.23.3 - UNRELEASED + + INCOMPATIBLE CHANGES + + HDFS-2676. Remove Avro RPC. (suresh) + + NEW FEATURES + + HDFS-2978. The NameNode should expose name dir statuses via JMX. (atm) + + HDFS-395. DFS Scalability: Incremental block reports. (Tomasz Nykiel + via hairong) + + HDFS-2517. Add protobuf service for JounralProtocol. (suresh) + + HDFS-2518. Add protobuf service for NamenodeProtocol. (suresh) + + HDFS-2520. Add protobuf service for InterDatanodeProtocol. (suresh) + + HDFS-2519. Add protobuf service for DatanodeProtocol. (suresh) + + HDFS-2581. Implement protobuf service for JournalProtocol. (suresh) + + HDFS-2618. Implement protobuf service for NamenodeProtocol. (suresh) + + HDFS-2629. Implement protobuf service for InterDatanodeProtocol. (suresh) + + HDFS-2636. Implement protobuf service for ClientDatanodeProtocol. (suresh) + + HDFS-2642. Protobuf translators for DatanodeProtocol. (jitendra) + + HDFS-2647. Used protobuf based RPC for InterDatanodeProtocol, + ClientDatanodeProtocol, JournalProtocol, NamenodeProtocol. (suresh) + + HDFS-2661. Enable protobuf RPC for DatanodeProtocol. (jitendra) + + HDFS-2697. Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings + protocol to protocol buffers. (jitendra) + + HDFS-2880. Protobuf changes in DatanodeProtocol to add multiple storages. + (suresh) + + HDFS-2899. Service protocol changes in DatanodeProtocol to add multiple + storages. (suresh) + + IMPROVEMENTS + + HDFS-2018. Move all journal stream management code into one place. + (Ivan Kelly via jitendra) + + HDFS-2223. Untangle depencencies between NN components (todd) + + HDFS-2351. Change Namenode and Datanode to register each of their protocols + seperately (sanjay) + + HDFS-2337. DFSClient shouldn't keep multiple RPC proxy references (atm) + + HDFS-2181. Separate HDFS Client wire protocol data types (sanjay) + + HDFS-2459. Separate datatypes for Journal Protocol. (suresh) + + HDFS-2480. Separate datatypes for NamenodeProtocol. (suresh) + + HDFS-2489. Move Finalize and Register to separate file out of + DatanodeCommand.java. (suresh) + + HDFS-2488. Separate datatypes for InterDatanodeProtocol. (suresh) + + HDFS-2496. Separate datatypes for DatanodeProtocol. (suresh) + + HDFS-2479. HDFS Client Data Types in Protocol Buffers (sanjay) + + HADOOP-7862. Hdfs changes to work with HADOOP-7862: Move the support for + multiple protocols to lower layer so that Writable, PB and Avro can all + use it. (sanjay) + + HDFS-2597. ClientNameNodeProtocol in Protocol Buffers. (sanjay) + + HDFS-2651. ClientNameNodeProtocol Translators for Protocol Buffers. (sanjay) + + HDFS-2650. Replace @inheritDoc with @Override. (Hari Mankude via suresh). + + HDFS-2669. Enable protobuf rpc for ClientNamenodeProtocol. (sanjay) + + HDFS-2801. Provide a method in client side translators to check for a + methods supported in underlying protocol. (jitendra) + + HDFS-2895. Remove Writable wire protocol types and translators to + complete transition to protocol buffers. (suresh) + + OPTIMIZATIONS + + BUG FIXES + + HDFS-2481. Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol. (sanjay) - HDFS-2497 Fix TestBackupNode failure. (suresh) + HDFS-2497. Fix TestBackupNode failure. (suresh) HDFS-2499. RPC client is created incorrectly introduced in HDFS-2459. (suresh) @@ -194,8 +217,9 @@ Trunk (unreleased changes) HDFS-2532. TestDfsOverAvroRpc timing out in trunk (Uma Maheswara Rao G via todd) - HDFS-1765. Block Replication should respect under-replication - block priority. (Uma Maheswara Rao G via eli) + HDFS-2666. Fix TestBackupNode failure. (suresh) + + HDFS-2663. Optional protobuf parameters are not handled correctly. (suresh) HDFS-2694. Removal of Avro broke non-PB NN services. (atm) @@ -205,39 +229,14 @@ Trunk (unreleased changes) HDFS-2700. Fix failing TestDataNodeMultipleRegistrations in trunk (Uma Maheswara Rao G via todd) - HDFS-2765. TestNameEditsConfigs is incorrectly swallowing IOE. (atm) - HDFS-2739. SecondaryNameNode doesn't start up. (jitendra) - HDFS-2776. Missing interface annotation on JournalSet. - (Brandon Li via jitendra) - HDFS-2768. BackupNode stop can not close proxy connections because it is not a proxy instance. (Uma Maheswara Rao G via eli) - HDFS-2759. Pre-allocate HDFS edit log files after writing version number. - (atm) - - HDFS-2908. Add apache license header for StorageReport.java. (Brandon Li - via jitendra) - HDFS-2968. Protocol translator for BlockRecoveryCommand broken when multiple blocks need recovery. (todd) -Release 0.23.3 - UNRELEASED - - INCOMPATIBLE CHANGES - - NEW FEATURES - - HDFS-2978. The NameNode should expose name dir statuses via JMX. (atm) - - IMPROVEMENTS - - OPTIMIZATIONS - - BUG FIXES - Release 0.23.2 - UNRELEASED INCOMPATIBLE CHANGES Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Feb 28 02:21:42 2012 @@ -1,4 +1,4 @@ -/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java:1159757-1294248 +/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java:1159757-1294443 /hadoop/core/branches/branch-0.19/hdfs/src/java:713112 /hadoop/core/branches/branch-0.19/hdfs/src/main/java:713112 /hadoop/core/trunk/src/hdfs:776175-785643,785929-786278 Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/native/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Feb 28 02:21:42 2012 @@ -1,4 +1,4 @@ -/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native:1152502-1294248 +/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native:1152502-1294443 /hadoop/core/branches/branch-0.19/hdfs/src/main/native:713112 /hadoop/core/branches/branch-0.19/mapred/src/c++/libhdfs:713112 /hadoop/core/trunk/src/c++/libhdfs:776175-784663 Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Feb 28 02:21:42 2012 @@ -1,4 +1,4 @@ -/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode:1159757-1294248 +/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode:1159757-1294443 /hadoop/core/branches/branch-0.19/hdfs/src/main/webapps/datanode:713112 /hadoop/core/branches/branch-0.19/hdfs/src/webapps/datanode:713112 /hadoop/core/trunk/src/webapps/datanode:776175-784663 Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Feb 28 02:21:42 2012 @@ -1,4 +1,4 @@ -/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs:1152502-1294248 +/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs:1152502-1294443 /hadoop/core/branches/branch-0.19/hdfs/src/main/webapps/hdfs:713112 /hadoop/core/branches/branch-0.19/hdfs/src/webapps/hdfs:713112 /hadoop/core/trunk/src/webapps/hdfs:776175-784663 Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Feb 28 02:21:42 2012 @@ -1,4 +1,4 @@ -/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary:1152502-1294248 +/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary:1152502-1294443 /hadoop/core/branches/branch-0.19/hdfs/src/main/webapps/secondary:713112 /hadoop/core/branches/branch-0.19/hdfs/src/webapps/secondary:713112 /hadoop/core/trunk/src/webapps/secondary:776175-784663 Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Feb 28 02:21:42 2012 @@ -1,4 +1,4 @@ -/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs:1159757-1294248 +/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs:1159757-1294443 /hadoop/core/branches/branch-0.19/hdfs/src/test/hdfs:713112 /hadoop/core/trunk/src/test/hdfs:776175-785643 /hadoop/hdfs/branches/HDFS-1052/src/test/hdfs:987665-1095512