hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6289) ROOT region doesn't get re-assigned in ServerShutdownHandler if the RS is still working but only the RS's ZK node expires.
Date Thu, 28 Jun 2012 18:04:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403307#comment-13403307
] 

stack commented on HBASE-6289:
------------------------------

bq. Any specific reason why we are verifying and then assigning the root region alone?

Are you asking why we assign the root, then the meta, ahead of all other assignments?  If
so, its because these need to be assigned for sure before any other assignments will complete.
 Maybe you were asking something else?
  
@Ted A test would be hard to get in here methinks for the startup code.  Would take a bunch
of mocking.  We, the hbase core, should make it easier on folks mocking up these scenarios
by building the necessary underpinnings before we can expect the likes of Maryann to deliver
a unit test (thats my opinion).
                
> ROOT region doesn't get re-assigned in ServerShutdownHandler if the RS is still working
but only the RS's ZK node expires.
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6289
>                 URL: https://issues.apache.org/jira/browse/HBASE-6289
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.90.6, 0.94.0
>            Reporter: Maryann Xue
>            Assignee: Maryann Xue
>            Priority: Critical
>         Attachments: HBASE-6289.patch
>
>
> The ROOT RS has some network problem and its ZK node expires first, which kicks off the
ServerShutdownHandler. it calls verifyAndAssignRoot() to try to re-assign ROOT. At that time,
the RS is actually still working and passes the verifyRootRegionLocation() check, so the ROOT
region is skipped from re-assignment.
> {code}
>   private void verifyAndAssignRoot()
>   throws InterruptedException, IOException, KeeperException {
>     long timeout = this.server.getConfiguration().
>       getLong("hbase.catalog.verification.timeout", 1000);
>     if (!this.server.getCatalogTracker().verifyRootRegionLocation(timeout)) {
>       this.services.getAssignmentManager().assignRoot();
>     }
>   }
> {code}
> After a few moments, this RS encounters DFS write problem and decides to abort. The RS
then soon gets restarted from commandline, and constantly report:
> {code}
> 2012-06-27 23:13:08,627 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException;
Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,627 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException;
Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,628 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException;
Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,628 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException;
Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,630 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException;
Region is not online: -ROOT-,,0
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message