Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 502569FB5 for ; Tue, 1 May 2012 22:23:18 +0000 (UTC) Received: (qmail 75751 invoked by uid 500); 1 May 2012 22:23:18 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 75708 invoked by uid 500); 1 May 2012 22:23:18 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 75698 invoked by uid 99); 1 May 2012 22:23:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 May 2012 22:23:18 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 May 2012 22:23:16 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id 56F5C42A88B for ; Tue, 1 May 2012 22:22:56 +0000 (UTC) Date: Tue, 1 May 2012 22:22:56 +0000 (UTC) From: "jiraposter@reviews.apache.org (JIRA)" To: issues@hbase.apache.org Message-ID: <1713654879.15186.1335910976363.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <1158881668.851.1335310925560.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (HBASE-5869) Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HBASE-5869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266195#comment-13266195 ] jiraposter@reviews.apache.org commented on HBASE-5869: ------------------------------------------------------ bq. On 2012-05-01 21:59:01, Ted Yu wrote: bq. > src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java, line 99 bq. > bq. > bq. > What if an AtomicInteger counter is added in the future ? Open new JIRA. This is just a move of existing code. bq. On 2012-05-01 21:59:01, Ted Yu wrote: bq. > src/main/java/org/apache/hadoop/hbase/SplitLogTask.java, line 155 bq. > bq. > bq. > What would the first 64 bytes of data represent ? bq. > Do we know that data.length >= 64 ? Let me fix toStringBinary so it deals w/ case where data is < 64 bytes. Regards what this represents, it could be anything. Just saving our logs from being filled w/ binary. bq. On 2012-05-01 21:59:01, Ted Yu wrote: bq. > src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java, line 567 bq. > bq. > bq. > Should e1 be included in the log ? bq. > 're-' before 'resubmit' is not necessary. Will add it. re- makes sense because this is retry inside exception handling. - Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4926/#review7446 ----------------------------------------------------------- On 2012-05-01 20:42:36, Michael Stack wrote: bq. bq. ----------------------------------------------------------- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/4926/ bq. ----------------------------------------------------------- bq. bq. (Updated 2012-05-01 20:42:36) bq. bq. bq. Review request for hbase and Jimmy Xiang. bq. bq. bq. Summary bq. ------- bq. bq. Convert two zk users to pb: distributed log splitting and regions in transition. bq. bq. Refactored distributed log splitting so we only serialize/deserialize in one location. bq. Less changes needed to do same for regions in transition. bq. bq. Moves serialization/deserialization out of the ZKAssign, ZKSplit and into bq. the classes themselves so can encapsulate how serialization is done into one place bq. (try to make the ZK* classes just deal in bytes -- about 90% done). bq. bq. Moved classes used by various packages up to top level to minimize imports bq. that are across package (zookeeper into protobuf and/or into regionserver and/or bq. master packages, etc). bq. bq. A src/main/java/org/apache/hadoop/hbase/DeserializationException.java bq. New generic deserialization exception. bq. A src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java bq. D src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java bq. Moved under zookeeper package. bq. A src/main/java/org/apache/hadoop/hbase/HBaseException.java bq. New base hbase exception as suggested by hbase-5796. New DeserializationException bq. inherits from this. bq. A src/main/java/org/apache/hadoop/hbase/RegionTransition.java bq. State of a region in transition. Top-level because used by a bq. few top-level packages. Encapsulates pb serialization/deserialization. bq. M src/main/java/org/apache/hadoop/hbase/ServerName.java bq. Add method to deserialize a ServeName, etc. Encapsulates pb'ing. bq. M src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java bq. Counters used by distributed log splitting. bq. A SplitLogTask bq. Class that encapsulates log splitting state. Also encapsulates pb'ing. bq. M src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java bq. Implement code for state. Added functions to go from code to state and vice bq. versa. Used serializing. bq. M src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java bq. Remove unused imports. bq. D src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java bq. Removed. Replaced by RegionTransition moved to package top-level. bq. M src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java bq. M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java bq. Use new DeserializationException. Move to using new RegionTransition bq. from RegionTransitionData class. Pass deserialized class rather than bq. byte array. Remove duplicated code. bq. M src/main/java/org/apache/hadoop/hbase/master/HMaster.java bq. Use new ServerName parse method rather than ZKUtil one. bq. M src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java bq. M src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java bq. Redo to use new SplitLogTask and SplitLogCounter classes. bq. M src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java bq. expectPBMagicPrefix added bq. M src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java bq. Use new RegionTransition in place of RegionTransitionData. bq. M src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java bq. Define moved from ZKSplitLog to SplitLogManager. bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java bq. Changed method name from getZNodeData to toByteArray to match how we've bq. named it elsewhere. Use new DeserializationException bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java bq. Use new RegionTransion class bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java bq. Moved stuff that was in here up into SplitLogManager where better bq. belongs. Also moved serialization/deserialization up into the bq. class itself: SplitLogTask. Moved counters out to SplitLogCounter class. bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bq. Moved deserialization of ServerName out of here and up into ServerName. bq. M src/main/protobuf/ZooKeeper.proto bq. Add two new classes, RegionTransition and SplitLogTask. bq. bq. bq. This addresses bug HBASE-5869. bq. https://issues.apache.org/jira/browse/HBASE-5869 bq. bq. bq. Diffs bq. ----- bq. bq. src/main/java/org/apache/hadoop/hbase/DeserializationException.java PRE-CREATION bq. src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java 9881ec2 bq. src/main/java/org/apache/hadoop/hbase/HBaseException.java PRE-CREATION bq. src/main/java/org/apache/hadoop/hbase/RegionTransition.java PRE-CREATION bq. src/main/java/org/apache/hadoop/hbase/ServerName.java 8fdb624 bq. src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java PRE-CREATION bq. src/main/java/org/apache/hadoop/hbase/SplitLogTask.java PRE-CREATION bq. src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 4121508 bq. src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java 06ca377 bq. src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java 35d7b70 bq. src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 47e3bd6 bq. src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java f56127d bq. src/main/java/org/apache/hadoop/hbase/master/HMaster.java 81e9023 bq. src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 692f194 bq. src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 919c65f bq. src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 994cb76 bq. src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 8457bdc bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java ebffad6 bq. src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java 8ea342f bq. src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java ea12da4 bq. src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 914b0d3 bq. src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 587386c bq. src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java PRE-CREATION bq. src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java f9575af bq. src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java babde80 bq. src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java e94b672 bq. src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java 30d7fe9 bq. src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 46a6fde bq. src/main/protobuf/ZooKeeper.proto 961ab65 bq. src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java c3a1889 bq. src/test/java/org/apache/hadoop/hbase/TestSerialization.java 50cb9d4 bq. src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java d496d48 bq. src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 1105ec9 bq. src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java 36dd289 bq. src/test/java/org/apache/hadoop/hbase/master/Mocking.java 676d6bb bq. src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 36046f8 bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 2669876 bq. src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 14cdb90 bq. src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java f8029ba bq. src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0f7d54e bq. src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 26b9865 bq. src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 75b5aea bq. src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java 07f8fc4 bq. src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java 55a8c4a bq. src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java 01dff76 bq. bq. Diff: https://reviews.apache.org/r/4926/diff bq. bq. bq. Testing bq. ------- bq. bq. bq. Thanks, bq. bq. Michael bq. bq. > Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb > ----------------------------------------------------------------------------------------------------- > > Key: HBASE-5869 > URL: https://issues.apache.org/jira/browse/HBASE-5869 > Project: HBase > Issue Type: Task > Reporter: stack > Assignee: stack > Attachments: 5869v7.txt, 5869v8.txt, 5869v9.txt, firstcut.txt, secondcut.txt, v10.txt, v11.txt, v12.txt, v13.txt, v4.txt, v5.txt, v6.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira