hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid
Date Wed, 18 Jan 2012 01:45:54 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188191#comment-13188191
] 

jiraposter@reviews.apache.org commented on HBASE-2600:
------------------------------------------------------



bq.  On 2012-01-17 19:54:30, Lars Hofhansl wrote:
bq.  > I love all that removed code!!
bq.  > I think it's fine to have it with patch (in fact removing all that code and the
main reason why we're doing this, right?)

We need this patch also to get rid of the perverse backtracking.


- Michael


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3466/#review4427
-----------------------------------------------------------


On 2012-01-18 01:22:37, Alex Newman wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3466/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-01-18 01:22:37)
bq.  
bq.  
bq.  Review request for hbase, Michael Stack and Lars Hofhansl.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  This is an idea that Ryan and I have been kicking around on and off for a while now.
bq.  
bq.  If regionnames were made of tablename+endrow instead of tablename+startrow, then in the
metatables, doing a search for the region that contains the wanted row, we'd just have to
open a scanner using passed row and the first row found by the scan would be that of the region
we need (If offlined parent, we'd have to scan to the next row).
bq.  
bq.  If we redid the meta tables in this format, we'd be using an access that is natural to
hbase, a scan as opposed to the perverse, expensive getClosestRowBefore we currently have
that has to walk backward in meta finding a containing region.
bq.  
bq.  This issue is about changing the way we name regions.
bq.  
bq.  If we were using scans, prewarming client cache would be near costless (as opposed to
what we'll currently have to do which is first a getClosestRowBefore and then a scan from
the closestrowbefore forward).
bq.  
bq.  Converting to the new method, we'd have to run a migration on startup changing the content
in meta.
bq.  
bq.  Up to this, the randomid component of a region name has been the timestamp of region
creation. HBASE-2531 "32-bit encoding of regionnames waaaaaaayyyyy too susceptible to hash
clashes" proposes changing the randomid so that it contains actual name of the directory in
the filesystem that hosts the region. If we had this in place, I think it would help with
the migration to this new way of doing the meta because as is, the region name in fs is a
hash of regionname... changing the format of the regionname would mean we generate a different
hash... so we'd need hbase-2531 to be in place before we could do this change.
bq.  
bq.  
bq.  This addresses bug HBASE-2600.
bq.      https://issues.apache.org/jira/browse/HBASE-2600
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/HConstants.java 8370ef8 
bq.    src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 74cb821 
bq.    src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 133759d 
bq.    src/main/java/org/apache/hadoop/hbase/KeyValue.java be7e2d8 
bq.    src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java e5e60a8 
bq.    src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 88c381f 
bq.    src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 99f90b2 
bq.    src/main/java/org/apache/hadoop/hbase/client/HTable.java aec7af2 
bq.    src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java 8af8c28 
bq.    src/main/java/org/apache/hadoop/hbase/client/HTablePool.java a5c198f 
bq.    src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java f0c6828 
bq.    src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 7a7b896 
bq.    src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java b47423c 
bq.    src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java c0a4184 
bq.    src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 25ae9a8 
bq.    src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 4307d89

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java
3a26bbb 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 48f6d77 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 79373e1 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java 50e7fe0 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java a3850e5

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 636e533 
bq.    src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java bf85bc1 
bq.    src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java f93c81d 
bq.    src/main/java/org/apache/hadoop/hbase/rest/model/TableRegionModel.java 67e7a04 
bq.    src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java 3fa5d41 
bq.    src/main/java/org/apache/hadoop/hbase/thrift/generated/Hbase.java 9e31c61 
bq.    src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift 5821d31 
bq.    src/test/java/org/apache/hadoop/hbase/TestKeyValue.java dc4ee8d 
bq.    src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 4b2b97c 
bq.    src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java dacb936

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java 5f97167

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java 6e1211b 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java 33c78ab 
bq.    src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java cffdcb6 
bq.    src/test/java/org/apache/hadoop/hbase/rest/model/TestTableRegionModel.java b6f0ab5

bq.  
bq.  Diff: https://reviews.apache.org/r/3466/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Unit tests started table. 
bq.  
bq.  
bq.  Tests in error: 
bq.    org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD: Table 'TestTable we searched
for the StartKey: TestTable ,, startKey lastChar's int value: 32 with the stopKey: TestTable#,,
stopRow lastChar's int value: 35 with parentTable:.META.
bq.  
bq.  I need to know how to update/recreate the tar ball which is the source for that test.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Alex
bq.  
bq.


                
> Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-2600
>                 URL: https://issues.apache.org/jira/browse/HBASE-2600
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Alex Newman
>         Attachments: 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
2600-trunk-01-17.txt, jenkins.pdf
>
>
> This is an idea that Ryan and I have been kicking around on and off for a while now.
> If regionnames were made of tablename+endrow instead of tablename+startrow, then in the
metatables, doing a search for the region that contains the wanted row, we'd just have to
open a scanner using passed row and the first row found by the scan would be that of the region
we need (If offlined parent, we'd have to scan to the next row).
> If we redid the meta tables in this format, we'd be using an access that is natural to
hbase, a scan as opposed to the perverse, expensive getClosestRowBefore we currently have
that has to walk backward in meta finding a containing region.
> This issue is about changing the way we name regions.
> If we were using scans, prewarming client cache would be near costless (as opposed to
what we'll currently have to do which is first a getClosestRowBefore and then a scan from
the closestrowbefore forward).
> Converting to the new method, we'd have to run a migration on startup changing the content
in meta.
> Up to this, the randomid component of a region name has been the timestamp of region
creation.   HBASE-2531 "32-bit encoding of regionnames waaaaaaayyyyy too susceptible to hash
clashes" proposes changing the randomid so that it contains actual name of the directory in
the filesystem that hosts the region.  If we had this in place, I think it would help with
the migration to this new way of doing the meta because as is, the region name in fs is a
hash of regionname... changing the format of the regionname would mean we generate a different
hash... so we'd need hbase-2531 to be in place before we could do this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message