hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5869) Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb
Date Tue, 01 May 2012 22:22:56 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266195#comment-13266195
] 

jiraposter@reviews.apache.org commented on HBASE-5869:
------------------------------------------------------



bq.  On 2012-05-01 21:59:01, Ted Yu wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java, line 99
bq.  > <https://reviews.apache.org/r/4926/diff/3/?file=105878#file105878line99>
bq.  >
bq.  >     What if an AtomicInteger counter is added in the future ?

Open new JIRA.  This is just a move of existing code.


bq.  On 2012-05-01 21:59:01, Ted Yu wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/SplitLogTask.java, line 155
bq.  > <https://reviews.apache.org/r/4926/diff/3/?file=105879#file105879line155>
bq.  >
bq.  >     What would the first 64 bytes of data represent ?
bq.  >     Do we know that data.length >= 64 ?

Let me fix toStringBinary so it deals w/ case where data is < 64 bytes.

Regards what this represents, it could be anything.  Just saving our logs from being filled
w/ binary.


bq.  On 2012-05-01 21:59:01, Ted Yu wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java, line 567
bq.  > <https://reviews.apache.org/r/4926/diff/3/?file=105887#file105887line567>
bq.  >
bq.  >     Should e1 be included in the log ?
bq.  >     're-' before 'resubmit' is not necessary.

Will add it.

re- makes sense because this is retry inside exception handling.


- Michael


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4926/#review7446
-----------------------------------------------------------


On 2012-05-01 20:42:36, Michael Stack wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4926/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-01 20:42:36)
bq.  
bq.  
bq.  Review request for hbase and Jimmy Xiang.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Convert two zk users to pb: distributed log splitting and regions in transition.
bq.  
bq.  Refactored distributed log splitting so we only serialize/deserialize in one location.
bq.  Less changes needed to do same for regions in transition.
bq.  
bq.  Moves serialization/deserialization out of the ZKAssign, ZKSplit and into
bq.  the classes themselves so can encapsulate how serialization is done into one place
bq.  (try to make the ZK* classes just deal in bytes -- about 90% done).
bq.  
bq.  Moved classes used by various packages up to top level to minimize imports
bq.  that are across package (zookeeper into protobuf and/or into regionserver and/or
bq.  master packages, etc).
bq.  
bq.  A src/main/java/org/apache/hadoop/hbase/DeserializationException.java
bq.    New generic deserialization exception.
bq.  A src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java
bq.  D  src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java
bq.    Moved under zookeeper package.
bq.  A src/main/java/org/apache/hadoop/hbase/HBaseException.java
bq.    New base hbase exception as suggested by hbase-5796.  New DeserializationException
bq.    inherits from this.
bq.  A src/main/java/org/apache/hadoop/hbase/RegionTransition.java
bq.    State of a region in transition.  Top-level because used by a
bq.    few top-level packages.  Encapsulates pb serialization/deserialization.
bq.  M src/main/java/org/apache/hadoop/hbase/ServerName.java
bq.    Add method to deserialize a ServeName, etc.  Encapsulates pb'ing.
bq.  M src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java
bq.    Counters used by distributed log splitting.
bq.  A SplitLogTask
bq.     Class that encapsulates log splitting state.  Also encapsulates pb'ing.
bq.  M src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
bq.    Implement code for state.  Added functions to go from code to state and vice
bq.    versa.  Used serializing.
bq.  M src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
bq.    Remove unused imports.
bq.  D src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java
bq.    Removed.  Replaced by RegionTransition moved to package top-level.
bq.  M src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
bq.  M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
bq.    Use new DeserializationException. Move to using new RegionTransition
bq.    from RegionTransitionData class.  Pass deserialized class rather than
bq.    byte array.  Remove duplicated code.
bq.  M src/main/java/org/apache/hadoop/hbase/master/HMaster.java
bq.    Use new ServerName parse method rather than ZKUtil one.
bq.  M src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
bq.  M src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java
bq.    Redo to use new SplitLogTask and SplitLogCounter classes.
bq.  M src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
bq.    expectPBMagicPrefix added
bq.  M src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
bq.    Use new RegionTransition in place of RegionTransitionData.
bq.  M src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
bq.    Define moved from ZKSplitLog to SplitLogManager.
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
bq.    Changed method name from getZNodeData to toByteArray to match how we've
bq.    named it elsewhere. Use new DeserializationException
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
bq.    Use new RegionTransion class
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java
bq.    Moved stuff that was in here up into SplitLogManager where better
bq.    belongs.  Also moved serialization/deserialization up into the
bq.    class itself: SplitLogTask.  Moved counters out to SplitLogCounter class.
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
bq.    Moved deserialization of ServerName out of here and up into ServerName.
bq.  M src/main/protobuf/ZooKeeper.proto
bq.    Add two new classes, RegionTransition and SplitLogTask.
bq.  
bq.  
bq.  This addresses bug HBASE-5869.
bq.      https://issues.apache.org/jira/browse/HBASE-5869
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/DeserializationException.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java 9881ec2 
bq.    src/main/java/org/apache/hadoop/hbase/HBaseException.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/RegionTransition.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/ServerName.java 8fdb624 
bq.    src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/SplitLogTask.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 4121508 
bq.    src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java 06ca377 
bq.    src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java 35d7b70 
bq.    src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 47e3bd6 
bq.    src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java f56127d 
bq.    src/main/java/org/apache/hadoop/hbase/master/HMaster.java 81e9023 
bq.    src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 692f194 
bq.    src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 919c65f 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 994cb76 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 8457bdc

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java ebffad6 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java 8ea342f 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java ea12da4 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 914b0d3 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 587386c 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java f9575af 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java babde80 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java e94b672 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java 30d7fe9 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 46a6fde 
bq.    src/main/protobuf/ZooKeeper.proto 961ab65 
bq.    src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java c3a1889 
bq.    src/test/java/org/apache/hadoop/hbase/TestSerialization.java 50cb9d4 
bq.    src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java d496d48 
bq.    src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 1105ec9 
bq.    src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java 36dd289 
bq.    src/test/java/org/apache/hadoop/hbase/master/Mocking.java 676d6bb 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 36046f8 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 2669876

bq.    src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 14cdb90 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java f8029ba 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0f7d54e 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 26b9865

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
75b5aea 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java
07f8fc4 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java
55a8c4a 
bq.    src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java 01dff76 
bq.  
bq.  Diff: https://reviews.apache.org/r/4926/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Michael
bq.  
bq.


                
> Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode
datas to pb 
> -----------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5869
>                 URL: https://issues.apache.org/jira/browse/HBASE-5869
>             Project: HBase
>          Issue Type: Task
>            Reporter: stack
>            Assignee: stack
>         Attachments: 5869v7.txt, 5869v8.txt, 5869v9.txt, firstcut.txt, secondcut.txt,
v10.txt, v11.txt, v12.txt, v13.txt, v4.txt, v5.txt, v6.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message