hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5453) Switch on-disk formats (reference files, HFile meta fields, etc) to PB
Date Thu, 17 May 2012 21:03:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278263#comment-13278263
] 

jiraposter@reviews.apache.org commented on HBASE-5453:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5130/#review7963
-----------------------------------------------------------

Ship it!



src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
<https://reviews.apache.org/r/5130/#comment17303>

    Your logic sounds correct and I agree we should avoid having to read the file each time.
    
    It would be cool to just see an example of the old writable vs new pb size to ensure our
intuition is correct.
    
    Worst case (let's say they happen to be the same) we could write out the new pb stuff
concatenated with the old way of writing and then they are guaranteed to be different sizes.
 Hopefully we don't have to do something like that, though.


- Gregory


On 2012-05-17 05:58:11, Michael Stack wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5130/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-17 05:58:11)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  A b/src/main/java/org/apache/hadoop/hbase/ClusterId.java
bq.    New  class to hold clusterid in.
bq.  M b/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
bq.    Make it so can do pb serialization.  Deprecated Writable serialization.
bq.  M b/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
bq.    Make it so methods in here follow the pattern in HCD an HTD pb 'ing.
bq.    Deprecated Writable serialization.
bq.  M b/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
bq.    Make it so can do pb serialization.  Deprecated Writable serialization.
bq.  M b/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
bq.    ClusterId under ZK got renamed as ZKClusterId
bq.  M b/src/main/java/org/apache/hadoop/hbase/io/Reference.java
bq.    Hide the Reference#Range enums.  Don't let them out of this class.
bq.    Make it so can do pb serialization.
bq.  M b/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
bq.    Use new methods on Reference for getting top and bottom.
bq.  M b/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
bq.    ClusterId under zk has been renamed ZKClusterId.
bq.    Use new ClusterId class too.
bq.  M b/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
bq.    Use new clusterid class.
bq.  M b/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
bq.    Move the RegionInfo convertion up into HRegionInfo instead of here.
bq.    Added generic toDelimitedByteArray helper.
bq.  M b/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
bq.    Use HRegionInfo convertions instead.
bq.  M b/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
bq.    Use HRegionInfo convertions instead.
bq.  M b/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
bq.    Use new utility writing out .regioninfo files.
bq.  M b/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
bq.    Formatting.
bq.  M b/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
bq.  M b/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
bq.    Range in Reference is no longer public.
bq.    Range in Reference is no longer public.
bq.  M b/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
bq.  M b/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenSecretManager.java
bq.    ClusterId got renamed ZKClusterId
bq.  M b/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
bq.    Use new serialization utlity in HTD.
bq.  M  b/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
bq.    Generic method for writing dot file content.
bq.  M b/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
bq.    Reference#Range is not public any more
bq.  M b/src/main/java/org/apache/hadoop/hbase/util/Writables.java
bq.    Deprecated getHRegionInfo, etc.
bq.  D b/src/main/java/org/apache/hadoop/hbase/zookeeper/ClusterId.java
bq.  A b/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java
bq.    Rename
bq.  A b/src/main/protobuf/ClusterId.proto
bq.    Added file for ClusterId only since its written to fs and to zk.
bq.  A b/src/main/protobuf/FS.proto
bq.    Protos for fs files.
bq.  M b/src/main/protobuf/ZooKeeper.proto
bq.    Moved ClusterId out to own proto file
bq.  M b/src/main/protobuf/hbase.proto
bq.    Added TableSchema and ColumnFamilySchema
bq.  
bq.  
bq.  This addresses bug hbase-5453.
bq.      https://issues.apache.org/jira/browse/hbase-5453
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/ClusterId.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java 5862f15 
bq.    src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 8d83ff3 
bq.    src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java af89e3e 
bq.    src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 5cac9af 
bq.    src/main/java/org/apache/hadoop/hbase/io/Reference.java 6360059 
bq.    src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 9e4ada9

bq.    src/main/java/org/apache/hadoop/hbase/master/HMaster.java 947ec5f 
bq.    src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 5052878 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java ccc964e 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java dabfbab 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java 45cb6cf 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 058c006 
bq.    src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 20c7738

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 9f16fee 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 6dc0517 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 6a9f2fe 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 5e1e16d 
bq.    src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
5050df0 
bq.    src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenSecretManager.java
049ed8d 
bq.    src/main/java/org/apache/hadoop/hbase/util/Bytes.java 2ccda17 
bq.    src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java efb2b84 
bq.    src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 3d35d3e 
bq.    src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 7b4f4a2 
bq.    src/main/java/org/apache/hadoop/hbase/util/Writables.java 3d20723 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ClusterId.java f804810 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java PRE-CREATION 
bq.    src/main/protobuf/ClusterId.proto PRE-CREATION 
bq.    src/main/protobuf/FS.proto PRE-CREATION 
bq.    src/main/protobuf/ZooKeeper.proto b72cb28 
bq.    src/main/protobuf/hbase.proto 30a4c3f 
bq.    src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptor.java e7fa8b2 
bq.    src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java f7c0cca 
bq.    src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 83d8408 
bq.    src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 69ccc65 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1020374 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java 6dfba41 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java 98c09eb

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 988d0bf 
bq.    src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 339a120 
bq.  
bq.  Diff: https://reviews.apache.org/r/5130/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Michael
bq.  
bq.


                
> Switch on-disk formats (reference files, HFile meta fields, etc) to PB
> ----------------------------------------------------------------------
>
>                 Key: HBASE-5453
>                 URL: https://issues.apache.org/jira/browse/HBASE-5453
>             Project: HBase
>          Issue Type: Sub-task
>          Components: ipc, master, migration, regionserver
>            Reporter: Todd Lipcon
>            Assignee: stack
>             Fix For: 0.96.0
>
>         Attachments: 5453.txt, 5453v10.txt, 5453v11.txt, 5453v11.txt, 5453v2.txt, 5453v3.txt,
5453v6.txt, 5453v9.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message