hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-19331) Region start-key/end-key corruption in Hbase meta table
Date Sat, 25 Nov 2017 05:29:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265591#comment-16265591
] 

Ted Yu edited comment on HBASE-19331 at 11/25/17 5:28 AM:
----------------------------------------------------------

I spent a bit time reviewing how mid key is calculated using branch-1.1
The code should be very similar between 0.98.x and 1.1

During splitting sample table, the following code in StoreFile#getFileSplitPoint() is executed:
{code}
      KeyValue mk = KeyValue.createKeyValueFromKey(midkey, 0, midkey.length);
{code}
When mk is not the same as first key and not the same as last key of the store file, mk.getRow()
is returned as split point.
The trailing zero is gone (as from e.g. \x00\x00\x00\x00\x00\x00\x07\x00). That is why the
assertion fails.

Did you observe problem accessing the split regions ?
I ask since \x00\x00\x00\x00\x00\x00\x07 can correctly separate the rows in the parent region.
Did you observe imbalanced number of rows in the daughter regions or other problems ?


was (Author: yuzhihong@gmail.com):
I spent a bit time reviewing how mid key is calculated using branch-1.1
The code should be very similar between 0.98.x and 1.1

During splitting sample table, the following code in StoreFile#getFileSplitPoint() is executed:
{code}
      KeyValue mk = KeyValue.createKeyValueFromKey(midkey, 0, midkey.length);
{code}
When mk is not the same as first key and not the same as last key of the store file, mk.getRow()
is returned as split point.
The trailing zero is gone (as in e.g. \x00\x00\x00\x00\x00\x00\x01\x00). That is why the assertion
fails.

Did you observe problem accessing the split regions ?
I ask since \x00\x00\x00\x00\x00\x00\x01 can correctly separate the rows in the parent region.
Did you observe imbalanced number of rows in the daughter regions or other problems ?

> Region start-key/end-key corruption in Hbase meta table
> -------------------------------------------------------
>
>                 Key: HBASE-19331
>                 URL: https://issues.apache.org/jira/browse/HBASE-19331
>             Project: HBase
>          Issue Type: Bug
>          Components: Region Assignment
>    Affects Versions: 0.98.8
>         Environment: Reproduced on HBase 0.98.8 on hadoop-2 
>            Reporter: Shamith kumar
>         Attachments: TestSplit.java
>
>
> when a region split happens on a key with trailing byte equals zero, the end key of the
first resulting region and and start key of the second resulting region in meta table gets
corrupted.
> Here is the link to code to reproduce this issue
> https://bitbucket.org/flytxt/hbase-meta-corruption-test
>  
> *+Test Result+*
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running com.flytxt.HbaseRegionMetaTest
> log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> 18:23:54.346 [main] INFO com.flytxt.HbaseRegionMetaTest - Dropping table SAMPLE_TBL_1
> 18:23:56.094 [main] INFO com.flytxt.HbaseRegionMetaTest - Dropping table SAMPLE_TBL_2
> 18:23:58.107 [main] INFO com.flytxt.HbaseRegionMetaTest - Creating new table SAMPLE_TBL_1
> 18:23:58.658 [main] INFO com.flytxt.HbaseRegionMetaTest - Creating new table SAMPLE_TBL_1
> 18:23:59.212 [main] INFO com.flytxt.HbaseRegionMetaTest - Starting puts to table SAMPLE_TBL_1
> 18:24:00.046 [main] INFO com.flytxt.HbaseRegionMetaTest - Puts complete .. lets split
SAMPLE_TBL_1
> 18:24:00.500 [main] INFO com.flytxt.HbaseRegionMetaTest - Starting puts to table SAMPLE_TBL_2
> 18:24:02.073 [main] INFO com.flytxt.HbaseRegionMetaTest - Puts complete .. lets split
SAMPLE_TBL_2
> 18:24:02.753 [main] INFO com.flytxt.HbaseRegionMetaTest - region split complete .. Lets
verify region infos for table SAMPLE_TBL_1 
> 18:24:02.754 [main] INFO com.flytxt.HbaseRegionMetaTest - ===========================================================
> 18:24:02.754 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Name : SAMPLE_TBL_1,,1511355240515.56c8fd8e42228c3c1ec71f9a4da65f5f.
> 18:24:02.755 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Id :1511355240515
> 18:24:02.755 [main] INFO com.flytxt.HbaseRegionMetaTest - Region start key :[]  , Key
length :0
> 18:24:02.755 [main] INFO com.flytxt.HbaseRegionMetaTest - Region end key : [0, 0, 0,
0, 0, 19, -76]  , Key length :7
> 18:24:02.755 [main] INFO com.flytxt.HbaseRegionMetaTest - -----------------------------------------------------------
> 18:24:02.762 [main] INFO com.flytxt.HbaseRegionMetaTest - ===========================================================
> 18:24:02.762 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Name : SAMPLE_TBL_1,\x00\x00\x00\x00\x00\x13\xB4,1511355240515.c06afed17b2a5c4fb54bacf704dd8a9e.
> 18:24:02.762 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Id :1511355240515
> 18:24:02.762 [main] INFO com.flytxt.HbaseRegionMetaTest - Region start key :[0, 0, 0,
0, 0, 19, -76]  , Key length :7
> 18:24:02.763 [main] INFO com.flytxt.HbaseRegionMetaTest - Region end key : []  , Key
length :0
> 18:24:02.763 [main] INFO com.flytxt.HbaseRegionMetaTest - -----------------------------------------------------------
> 18:24:03.005 [main] INFO com.flytxt.HbaseRegionMetaTest - region split complete .. Lets
verify region infos for table SAMPLE_TBL_2 
> 18:24:03.006 [main] INFO com.flytxt.HbaseRegionMetaTest - ===========================================================
> 18:24:03.006 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Name : SAMPLE_TBL_2,,1511355242363.0679851100e16aad005c743af618452e.
> 18:24:03.006 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Id :1511355242363
> 18:24:03.007 [main] INFO com.flytxt.HbaseRegionMetaTest - Region start key :[]  , Key
length :0
> 18:24:03.008 [main] INFO com.flytxt.HbaseRegionMetaTest - Region end key : [0, 0, 0,
0, 0, 0, 19, -57]  , Key length :8
> 18:24:03.008 [main] INFO com.flytxt.HbaseRegionMetaTest - -----------------------------------------------------------
> 18:24:03.008 [main] INFO com.flytxt.HbaseRegionMetaTest - ===========================================================
> 18:24:03.008 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Name : SAMPLE_TBL_2,\x00\x00\x00\x00\x00\x00\x13\xC7,1511355242363.326b5175036efdeae89f9493b436e114.
> 18:24:03.008 [main] INFO com.flytxt.HbaseRegionMetaTest - Region Id :1511355242363
> 18:24:03.008 [main] INFO com.flytxt.HbaseRegionMetaTest - Region start key :[0, 0, 0,
0, 0, 0, 19, -57]  , Key length :8
> 18:24:03.008 [main] INFO com.flytxt.HbaseRegionMetaTest - Region end key : []  , Key
length :0
> 18:24:03.009 [main] INFO com.flytxt.HbaseRegionMetaTest - -----------------------------------------------------------
> [ERROR] Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 10.777 s <<<
FAILURE! - in com.flytxt.HbaseRegionMetaTest
> [ERROR] testTableRegionStartAndEndKey[0](com.flytxt.HbaseRegionMetaTest)  Time elapsed:
0.228 s  <<< FAILURE!
> java.lang.AssertionError: 
> Region end key error for SAMPLE_TBL_1,,1511355240515.56c8fd8e42228c3c1ec71f9a4da65f5f.
> Expected: is <8>
>      but: was <7>
> 	at com.flytxt.HbaseRegionMetaTest.testTableRegionStartAndEndKey(HbaseRegionMetaTest.java:170)
> [ERROR] testTableRegionStartAndEndKey[0](com.flytxt.HbaseRegionMetaTest)  Time elapsed:
0.23 s  <<< FAILURE!
> java.lang.AssertionError: 
> Region start key error for SAMPLE_TBL_1,\x00\x00\x00\x00\x00\x13\xB4,1511355240515.c06afed17b2a5c4fb54bacf704dd8a9e.
> Expected: is <8>
>      but: was <7>
> 	at com.flytxt.HbaseRegionMetaTest.testTableRegionStartAndEndKey(HbaseRegionMetaTest.java:167)
> [INFO] 
> [INFO] Results:
> [INFO] 
> [ERROR] Failures: 
> [ERROR] com.flytxt.HbaseRegionMetaTest.testTableRegionStartAndEndKey[0](com.flytxt.HbaseRegionMetaTest)
> [ERROR]   Run 1: HbaseRegionMetaTest.testTableRegionStartAndEndKey:170 Region end key
error for SAMPLE_TBL_1,,1511355240515.56c8fd8e42228c3c1ec71f9a4da65f5f.
> Expected: is <8>
>      but: was <7>
> [ERROR]   Run 2: HbaseRegionMetaTest.testTableRegionStartAndEndKey:167 Region start key
error for SAMPLE_TBL_1,\x00\x00\x00\x00\x00\x13\xB4,1511355240515.c06afed17b2a5c4fb54bacf704dd8a9e.
> Expected: is <8>
>      but: was <7>
> [INFO] 
> [INFO] 
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
>   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message