hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4842) [hbck] Fix intermittent failures on TestHBaseFsck.testHBaseFsck
Date Tue, 22 Nov 2011 01:06:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154788#comment-13154788
] 

Jonathan Hsieh commented on HBASE-4842:
---------------------------------------

The story behind this problem.

HBCK repairs a bad assignment using the admin interface to reassign a particular region. 
 offlining the region in ZK.  This calls master.assign -- eventually the master uses its serverManager
and issues an HRegionServer.openRegion().

Looks like the HRegionServer.openRegion being essentially asynchronous and causes the failure.
 The call submits an OpenRegionHandler (ORH) callback to the RS's ExecutorService and then
immediately returns the RegionState to OPENED.

The ORH thread calls ORH.process -> updateMeta, which creates a PostOpenDeployTaskThread
and starts another thread that calls  HRegionServer.postOpenDeployTasks -> MetaEditor.updateRegionLocation
which updates the meta table.  

The problem is that the RegionState OPENED is reported to the master even though it may not
have written all its new assignment to META yet.



                
> [hbck] Fix intermittent failures on TestHBaseFsck.testHBaseFsck
> ---------------------------------------------------------------
>
>                 Key: HBASE-4842
>                 URL: https://issues.apache.org/jira/browse/HBASE-4842
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jonathan Hsieh
>
> Its seems that on the 0.92 branch in particular, TestHBaseFsck.testHBaseFsck is intermittently
failing.
> In the test, a region's assignment is purposely changed in META but not in ZK.  After
the equivalent of 'hbck -fix', a subsequent check that should be clean comes up with a new
ZK assignment but with META still being inconsistent with ZK.  The RS in ZK sometimes this
points to the same RS, but sometimes it "moves" to another ZK. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message