hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2244) META gets inconsistent in a number of crash scenarios
Date Wed, 03 Mar 2010 04:53:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12840458#action_12840458
] 

stack commented on HBASE-2244:
------------------------------

@Kannan I committed below to address your last comment.  Let me know if insufficient.

{code}
Index: core/src/main/java/org/apache/hadoop/hbase/master/BaseScanner.java
===================================================================
--- core/src/main/java/org/apache/hadoop/hbase/master/BaseScanner.java  (revision 918324)
+++ core/src/main/java/org/apache/hadoop/hbase/master/BaseScanner.java  (working copy)
@@ -342,9 +342,8 @@
     if (!references) return references;
     if (!verifyDaughterRowPresent(rowContent, qualifier, srvr, metaRegionName,
         hri, parent)) {
-      // If we got here, we added a daughter region to metatable. Update
-      // parent row that daughter has been verified present so we don't check
-      // for it by doing a get each time through here.
+      // If we got here, then the parent row does not yet have the
+      // "daughter row verified present" marker present. Add it.
       addDaughterRowChecked(metaRegionName, srvr, parent.getRegionName(), hri,
         qualifier);
     }
@@ -360,8 +359,8 @@
    * @param metaRegionName
    * @param daughterHRI
    * @throws IOException
-   * @return True, if the daughter row is present in meta.  If false, this
-   * method just added it to meta.
+   * @return True, if parent row has marker for "daughter row verified present"
+   * else, false (and will do fixup adding daughter if daughter not present).
    */
   private boolean verifyDaughterRowPresent(final Result rowContent,
       final byte [] daughter, final HRegionInterface srvr,
{code}

> META gets inconsistent in a number of crash scenarios
> -----------------------------------------------------
>
>                 Key: HBASE-2244
>                 URL: https://issues.apache.org/jira/browse/HBASE-2244
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: Kannan Muthukkaruppan
>            Assignee: stack
>            Priority: Critical
>             Fix For: 0.20.4
>
>         Attachments: 2244-v3.patch, 2244-v5.patch, 2244-v6.patch, 2244.patch
>
>
> (Forking this issue off from HBASE-2235).
> During load testing, in a number of failure scenarios (unexpected region server deaths)
etc., we notice that META can get inconsistent. This primarily happens for regions which are
in the process of being split. Manually running add_table.rb seems to fix the tables meta
data just fine. 
> But it would be good to do automatic cleansing (as part of META scanners work) and/or
avoid these inconsistent states altogether.
> For example, for a particular startkey, I see all these entries:
> {code}
> test1,1204765,1266569946560 column=info:regioninfo, timestamp=1266581302018, value=REGION
=> {NAME => 'test1,
>                              1204765,1266569946560', STARTKEY => '1204765', ENDKEY
=> '1441091', ENCODED => 18
>                              19368969, OFFLINE => true, SPLIT => true, TABLE =>
{{NAME => 'test1', FAMILIES =>
>                               [{NAME => 'actions', VERSIONS => '3', COMPRESSION
=> 'NONE', TTL => '2147483647'
>                              , BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE
=> 'true'}]}}
>  test1,1204765,1266569946560 column=info:server, timestamp=1266570029133, value=10.129.68.212:60020
>  test1,1204765,1266569946560 column=info:serverstartcode, timestamp=1266570029133, value=1266562597546
>  test1,1204765,1266569946560 column=info:splitB, timestamp=1266581302018, value=\x00\x071441091\x00\x00\x00\x0
>                              1\x26\xE6\x1F\xDF\x27\x1Btest1,1290703,1266581233447\x00\x071290703\x00\x00\x00\x
>                              05\x05test1\x00\x00\x00\x00\x00\x02\x00\x00\x00\x07IS_ROOT\x00\x00\x00\x05false\x
>                              00\x00\x00\x07IS_META\x00\x00\x00\x05false\x00\x00\x00\x01\x07\x07actions\x00\x00
>                              \x00\x07\x00\x00\x00\x0BBLOOMFILTER\x00\x00\x00\x05false\x00\x00\x00\x0BCOMPRESSI
>                              ON\x00\x00\x00\x04NONE\x00\x00\x00\x08VERSIONS\x00\x00\x00\x013\x00\x00\x00\x03TT
>                              L\x00\x00\x00\x0A2147483647\x00\x00\x00\x09BLOCKSIZE\x00\x00\x00\x0565536\x00\x00
>                              \x00\x09IN_MEMORY\x00\x00\x00\x05false\x00\x00\x00\x0ABLOCKCACHE\x00\x00\x00\x04t
>                              rueh\x0FQ\xCF
>  test1,1204765,1266581233447 column=info:regioninfo, timestamp=1266609172177, value=REGION
=> {NAME => 'test1,
>                              1204765,1266581233447', STARTKEY => '1204765', ENDKEY
=> '1290703', ENCODED => 13
>                              73493090, OFFLINE => true, SPLIT => true, TABLE =>
{{NAME => 'test1', FAMILIES =>
>                               [{NAME => 'actions', VERSIONS => '3', COMPRESSION
=> 'NONE', TTL => '2147483647'
>                              , BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE
=> 'true'}]}}
>  test1,1204765,1266581233447 column=info:server, timestamp=1266604768670, value=10.129.68.213:60020
>  test1,1204765,1266581233447 column=info:serverstartcode, timestamp=1266604768670, value=1266562597511
>  test1,1204765,1266581233447 column=info:splitA, timestamp=1266609172177, value=\x00\x071226169\x00\x00\x00\x0
>                              1\x26\xE7\xCA,\x7D\x1Btest1,1204765,1266609171581\x00\x071204765\x00\x00\x00\x05\
>                              x05test1\x00\x00\x00\x00\x00\x02\x00\x00\x00\x07IS_ROOT\x00\x00\x00\x05false\x00\
>                              x00\x00\x07IS_META\x00\x00\x00\x05false\x00\x00\x00\x01\x07\x07actions\x00\x00\x0
>                              0\x07\x00\x00\x00\x0BBLOOMFILTER\x00\x00\x00\x05false\x00\x00\x00\x0BCOMPRESSION\
>                              x00\x00\x00\x04NONE\x00\x00\x00\x08VERSIONS\x00\x00\x00\x013\x00\x00\x00\x03TTL\x
>                              00\x00\x00\x0A2147483647\x00\x00\x00\x09BLOCKSIZE\x00\x00\x00\x0565536\x00\x00\x0
>                              0\x09IN_MEMORY\x00\x00\x00\x05false\x00\x00\x00\x0ABLOCKCACHE\x00\x00\x00\x04true
>                              \xB9\xBD\xFEO
>  test1,1204765,1266581233447 column=info:splitB, timestamp=1266609172177, value=\x00\x071290703\x00\x00\x00\x0
>                              1\x26\xE7\xCA,\x7D\x1Btest1,1226169,1266609171581\x00\x071226169\x00\x00\x00\x05\
>                              x05test1\x00\x00\x00\x00\x00\x02\x00\x00\x00\x07IS_ROOT\x00\x00\x00\x05false\x00\
>                              x00\x00\x07IS_META\x00\x00\x00\x05false\x00\x00\x00\x01\x07\x07actions\x00\x00\x0
>                              0\x07\x00\x00\x00\x0BBLOOMFILTER\x00\x00\x00\x05false\x00\x00\x00\x0BCOMPRESSION\
>                              x00\x00\x00\x04NONE\x00\x00\x00\x08VERSIONS\x00\x00\x00\x013\x00\x00\x00\x03TTL\x
>                              00\x00\x00\x0A2147483647\x00\x00\x00\x09BLOCKSIZE\x00\x00\x00\x0565536\x00\x00\x0
>                              0\x09IN_MEMORY\x00\x00\x00\x05false\x00\x00\x00\x0ABLOCKCACHE\x00\x00\x00\x04true
>                              \xE1\xDF\xF8p
>  test1,1204765,1266609171581 column=info:regioninfo, timestamp=1266609172212, value=REGION
=> {NAME => 'test1,
>                              1204765,1266609171581', STARTKEY => '1204765', ENDKEY
=> '1226169', ENCODED => 21
>                              34878372, TABLE => {{NAME => 'test1', FAMILIES =>
[{NAME => 'actions', VERSIONS =
>                              > '3', COMPRESSION => 'NONE', TTL => '2147483647',
BLOCKSIZE => '65536', IN_MEMOR
>                              Y => 'false', BLOCKCACHE => 'true'}]}}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message