hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3381) Interrupt of a region open comes across as a successful open
Date Wed, 22 Dec 2010 17:44:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974295#action_12974295
] 

stack commented on HBASE-3381:
------------------------------

Here is evidence the mechanism is basically working (though this is not really what ths specific
issue is about..... still need evidence that that is working):

{code}
2010-12-22 17:17:25,352 DEBUG org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Interrupting thread Thread[PostOpenDeployTasks:9872342bec238398a94dee894ff29a6f,5,main]
2010-12-22 17:17:25,352 WARN org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Exception running postOpenDeployTasks; region=9872342bec238398a94dee894ff29a6f
org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Interrupted
    at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:364)
    at org.apache.hadoop.hbase.catalog.MetaEditor.updateRegionLocation(MetaEditor.java:146)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1331)
    at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:205)
2010-12-22 17:17:25,352 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing TestTable,0612723473,1293034036869.9872342bec238398a94dee894ff29a6f.:
disabling compactions & flushes
2010-12-22 17:17:25,352 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled
for region TestTable,0612723473,1293034036869.9872342bec238398a94dee894ff29a6f.
2010-12-22 17:17:25,352 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed info
...
{code}


> Interrupt of a region open comes across as a successful open
> ------------------------------------------------------------
>
>                 Key: HBASE-3381
>                 URL: https://issues.apache.org/jira/browse/HBASE-3381
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>             Fix For: 0.90.0
>
>         Attachments: 3381.txt
>
>
> Meta was offline when below happened:
> {code}
> 2010-12-21 19:45:23,023 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12d0a53c540000e
Attempting to transition node 337038b50e467fbd6b031f278bbd9c22 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENING
> 2010-12-21 19:45:23,046 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12d0a53c540000e
Successfully transitioned node 337038b50e467fbd6b031f278bbd9c22 from RS_ZK_REGION_OPENING
to RS_ZK_REGION_OPENING
> 2010-12-21 19:45:26,379 DEBUG org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Interrupting thread Thread[PostOpenDeployTasks:337038b50e467fbd6b031f278bbd9c22,5,main]
> 2010-12-21 19:45:26,379 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12d0a53c540000e
Attempting to transition node 337038b50e467fbd6b031f278bbd9c22 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
> 2010-12-21 19:45:26,381 WARN org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Exception running postOpenDeployTasks; region=337038b50e467fbd6b031f278bbd9c22
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Interrupted
>     at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:364)
>     at org.apache.hadoop.hbase.catalog.MetaEditor.updateRegionLocation(MetaEditor.java:146)
>     at org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1331)
>     at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:195)
> ...
> {code}
> So, we timed out trying to open the region but rather than close the region because edit
failed, we missed seeing the InterruptedException.
> Here is suggested fix:
> {code}
> diff --git a/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java b/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
> index 7bf680d..2b0078c 100644
> --- a/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
> +++ b/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
> @@ -339,7 +339,7 @@ public class MetaReader {
>      get.addFamily(HConstants.CATALOG_FAMILY);
>      byte [] meta = getCatalogRegionNameForRegion(regionName);
>      Result r = catalogTracker.waitForMetaServerConnectionDefault().get(meta, get);
> -    if(r == null || r.isEmpty()) {
> +    if (r == null || r.isEmpty()) {
>        return null;
>      }
>      return metaRowToRegionPair(r);
> {code}
> Let me try it.
> W/o this, what we see is hbck showing that region is on server X but in .META. it shows
as being on Y (its pre-balance server)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message