hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5869) Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb
Date Sat, 28 Apr 2012 22:20:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264419#comment-13264419

jiraposter@reviews.apache.org commented on HBASE-5869:

bq.  On 2012-04-28 22:14:23, Jimmy Xiang wrote:
bq.  > src/main/protobuf/ZooKeeper.proto, line 82
bq.  > <https://reviews.apache.org/r/4926/diff/1/?file=105372#file105372line82>
bq.  >
bq.  >     A task is a path, this is more like a task state, isn't it?

I can change this np.

Currently I have the pb class named same as the class that wraps it.  Should I change this?
 Add a pb prefix or something?   Problem w/ that is that no other of the pb classes have the
pb prefix.  They are in the generated package which is probably sufficient to distingush them?
 My hope is to make it so the pbs do not leak outside of the class that serializes to them;
e.g. this SplitLogTask class.

bq.  On 2012-04-28 22:14:23, Jimmy Xiang wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java, line 798
bq.  > <https://reviews.apache.org/r/4926/diff/1/?file=105369#file105369line798>
bq.  >
bq.  >      I see this many places. Can we put the whole block in a util method? Probably
will be easier to read?

Will do.

bq.  On 2012-04-28 22:14:23, Jimmy Xiang wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java, line 182
bq.  > <https://reviews.apache.org/r/4926/diff/1/?file=105357#file105357line182>
bq.  >
bq.  >     Should we abort? Under what scenario the parsing can fail, other than a conflict
data format?

I thought I was just redoing what was there previous.  We could abort but maybe next time
through the deserialization works because its been updated by another?  Or, we spew this error
all over the logs and drive someone crazy?   Will look at it again.

- Michael

This is an automatically generated e-mail. To reply, visit:

On 2012-04-28 18:10:24, Michael Stack wrote:
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4926/
bq.  -----------------------------------------------------------
bq.  (Updated 2012-04-28 18:10:24)
bq.  Review request for hbase and Jimmy Xiang.
bq.  Summary
bq.  -------
bq.  Convert two zk users to pb: distributed log splitting and regions in transition.
bq.  Refactored distributed log splitting so we only serialize/deserialize in one location.
bq.  Less changes needed to do same for regions in transition.
bq.  Moves serialization/deserialization out of the ZKAssign, ZKSplit and into
bq.  the classes themselves so can encapsulate how serialization is done into one place
bq.  (try to make the ZK* classes just deal in bytes -- about 90% done).
bq.  Moved classes used by various packages up to top level to minimize imports
bq.  that are across package (zookeeper into protobuf and/or into regionserver and/or
bq.  master packages, etc).
bq.  A src/main/java/org/apache/hadoop/hbase/DeserializationException.java
bq.    New generic deserialization exception.
bq.  A src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java
bq.  D  src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java
bq.    Moved under zookeeper package.
bq.  A src/main/java/org/apache/hadoop/hbase/HBaseException.java
bq.    New base hbase exception as suggested by hbase-5796.  New DeserializationException
bq.    inherits from this.
bq.  A src/main/java/org/apache/hadoop/hbase/RegionTransition.java
bq.    State of a region in transition.  Top-level because used by a
bq.    few top-level packages.  Encapsulates pb serialization/deserialization.
bq.  M src/main/java/org/apache/hadoop/hbase/ServerName.java
bq.    Add method to deserialize a ServeName, etc.  Encapsulates pb'ing.
bq.  M src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java
bq.    Counters used by distributed log splitting.
bq.  A SplitLogTask
bq.     Class that encapsulates log splitting state.  Also encapsulates pb'ing.
bq.  M src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
bq.    Implement code for state.  Added functions to go from code to state and vice
bq.    versa.  Used serializing.
bq.  M src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
bq.    Remove unused imports.
bq.  D src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java
bq.    Removed.  Replaced by RegionTransition moved to package top-level.
bq.  M src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
bq.  M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
bq.    Use new DeserializationException. Move to using new RegionTransition
bq.    from RegionTransitionData class.  Pass deserialized class rather than
bq.    byte array.  Remove duplicated code.
bq.  M src/main/java/org/apache/hadoop/hbase/master/HMaster.java
bq.    Use new ServerName parse method rather than ZKUtil one.
bq.  M src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
bq.  M src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java
bq.    Redo to use new SplitLogTask and SplitLogCounter classes.
bq.  M src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
bq.    expectPBMagicPrefix added
bq.  M src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
bq.    Use new RegionTransition in place of RegionTransitionData.
bq.  M src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
bq.    Define moved from ZKSplitLog to SplitLogManager.
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
bq.    Changed method name from getZNodeData to toByteArray to match how we've
bq.    named it elsewhere. Use new DeserializationException
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
bq.    Use new RegionTransion class
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java
bq.    Moved stuff that was in here up into SplitLogManager where better
bq.    belongs.  Also moved serialization/deserialization up into the
bq.    class itself: SplitLogTask.  Moved counters out to SplitLogCounter class.
bq.  M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
bq.    Moved deserialization of ServerName out of here and up into ServerName.
bq.  M src/main/protobuf/ZooKeeper.proto
bq.    Add two new classes, RegionTransition and SplitLogTask.
bq.  This addresses bug HBASE-5869.
bq.      https://issues.apache.org/jira/browse/HBASE-5869
bq.  Diffs
bq.  -----
bq.    src/main/java/org/apache/hadoop/hbase/DeserializationException.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java 9881ec2 
bq.    src/main/java/org/apache/hadoop/hbase/HBaseException.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/RegionTransition.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/ServerName.java 8fdb624 
bq.    src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/SplitLogTask.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 4121508 
bq.    src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java 06ca377 
bq.    src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java 35d7b70 
bq.    src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 47e3bd6 
bq.    src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java f56127d 
bq.    src/main/java/org/apache/hadoop/hbase/master/HMaster.java 81e9023 
bq.    src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 919c65f 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 994cb76 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 8457bdc

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java 8ea342f 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java ea12da4 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 587386c 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java f9575af 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java babde80 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java e94b672 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java 30d7fe9 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 46a6fde 
bq.    src/main/protobuf/ZooKeeper.proto 961ab65 
bq.    src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java c3a1889 
bq.    src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java d496d48 
bq.    src/test/java/org/apache/hadoop/hbase/master/Mocking.java 676d6bb 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 36046f8 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 2669876

bq.    src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 14cdb90 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java f8029ba 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0f7d54e 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 26b9865

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java
bq.    src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java 4314572 
bq.  Diff: https://reviews.apache.org/r/4926/diff
bq.  Testing
bq.  -------
bq.  Thanks,
bq.  Michael

> Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode
datas to pb 
> -----------------------------------------------------------------------------------------------------
>                 Key: HBASE-5869
>                 URL: https://issues.apache.org/jira/browse/HBASE-5869
>             Project: HBase
>          Issue Type: Task
>            Reporter: stack
>            Assignee: stack
>         Attachments: firstcut.txt, secondcut.txt, v4.txt, v5.txt, v6.txt

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message