hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5986) Clients can see holes in the META table when regions are being split
Date Thu, 17 May 2012 02:16:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277494#comment-13277494
] 

jiraposter@reviews.apache.org commented on HBASE-5986:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5133/#review7948
-----------------------------------------------------------


v2 looks much better.


src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
<https://reviews.apache.org/r/5133/#comment17287>

    We may need to mark this JIRA an incompatible change.



src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
<https://reviews.apache.org/r/5133/#comment17283>

    'with' -> 'by'



src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
<https://reviews.apache.org/r/5133/#comment17284>

    This class can be private.



src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
<https://reviews.apache.org/r/5133/#comment17285>

    The interval can be made shorter.



src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
<https://reviews.apache.org/r/5133/#comment17286>

    Name this method verifyRegionsOfHTable ?


- Ted


On 2012-05-17 00:58:21, enis wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5133/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-17 00:58:21)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  We found this issue when running large scale ingestion tests for HBASE-5754. The problem
is that the .META. table updates are not atomic while splitting a region. In SplitTransaction,
there is a time lap between the marking the parent offline, and adding of daughters to the
META table. This can result in clients using MetaScanner, of HTable.getStartEndKeys (used
by the TableInputFormat) missing regions which are made just offline, but the daughters are
not added yet.
bq.  
bq.  This patch is the approach 2 mentioned in the issue comments, mainly during META scan,
if we detect that the region is split, we block until the information for the child regions
are available in META and manually feed those rows to the MetaScanner. Although approach 3
(using local region transactions) seems cleaner, they are not available under branch 0.92,
which I think should also incorporate this fix. I'll provide ports once we are clear for trunk.

bq.  
bq.  Also this patch does not fix MetaReader (see https://issues.apache.org/jira/browse/HBASE-3475).

bq.  
bq.  
bq.  This addresses bug HBASE-5986.
bq.      https://issues.apache.org/jira/browse/HBASE-5986
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java 8873512 
bq.    src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 5d4be3f 
bq.    src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 5cac9af 
bq.    src/main/java/org/apache/hadoop/hbase/client/HTable.java b8290e4 
bq.    src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java f404999 
bq.    src/main/java/org/apache/hadoop/hbase/master/HMaster.java 0ad9b18 
bq.    src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 7b4f4a2 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
a8091e6 
bq.  
bq.  Diff: https://reviews.apache.org/r/5133/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  added extensive tests under TestEndToEndSplitTranscation, and ran existing unit tests.

bq.  
bq.  
bq.  Thanks,
bq.  
bq.  enis
bq.  
bq.


                
> Clients can see holes in the META table when regions are being split
> --------------------------------------------------------------------
>
>                 Key: HBASE-5986
>                 URL: https://issues.apache.org/jira/browse/HBASE-5986
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.92.1, 0.96.0, 0.94.1
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>         Attachments: 5986-v2.txt, HBASE-5986-test_v1.patch
>
>
> We found this issue when running large scale ingestion tests for HBASE-5754. The problem
is that the .META. table updates are not atomic while splitting a region. In SplitTransaction,
there is a time lap between the marking the parent offline, and adding of daughters to the
META table. This can result in clients using MetaScanner, of HTable.getStartEndKeys (used
by the TableInputFormat) missing regions which are made just offline, but the daughters are
not added yet. 
> This is also related to HBASE-4335. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message