hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeffrey Zhong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
Date Thu, 21 Mar 2013 18:27:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609265#comment-13609265
] 

Jeffrey Zhong commented on HBASE-8127:
--------------------------------------

[~rajesh23] If you run the test in a loop, you most likely will see it fail about once in
5 times due to the reason I logged in https://issues.apache.org/jira/browse/HBASE-8169. 

After I apply your fix + repro_hang + the fix of hbase-8169 on the test case, I was able to
run the test successfully 20 times in a loop.(I saw it failed once due to RS could not start,
which I think it's normal to other test cases as well).

If you like, you can reassign the JIRA to yourself.
                
> Region of a disabling or disabled table could be stuck in transition state when RS dies
during Master initialization
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-8127
>                 URL: https://issues.apache.org/jira/browse/HBASE-8127
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.94.5
>            Reporter: Jeffrey Zhong
>            Assignee: Jeffrey Zhong
>             Fix For: 0.94.7
>
>         Attachments: HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
>
>
> The issue happens when a RS dies during a master starts up. After the RS reports open
to the new master instance and dies immediately thereafter, the RITs of disabling tables(or
disabled table) on the died RS will be in RIT state forever.
> I attached a patch to simulate the situation and you can run the following command to
reproduce the issue:
> {code}mvn test -PlocalTests -Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
> Basically, we skip regions of a dead server inside AM.processDeadServersAndRecoverLostRegions
as the following code and relies on SSH to process those skipped regions:
> {code}
>           for (Pair<HRegionInfo, Result> deadRegion : deadServer.getValue()) {
>             nodes.remove(deadRegion.getFirst().getEncodedName());
>           }
> {code} 
> While in SSH, we skip regions of disabling(or disabled table) again by function processDeadRegion.
Finally comes to the issue that RITs of disabling(or disabled table) stuck there forever.
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message