hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6300) Master should not ignore event RS_ZK_REGION_OPENED when regionState is null or unexpected.
Date Mon, 02 Jul 2012 12:35:22 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405040#comment-13405040
] 

stack commented on HBASE-6300:
------------------------------

Sounds good Maryann.  Why do you think RegionState was null?  Did something happen?  Let me
look at hbase-6299.
                
> Master should not ignore event RS_ZK_REGION_OPENED when regionState is null or unexpected.
> ------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6300
>                 URL: https://issues.apache.org/jira/browse/HBASE-6300
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.90.6, 0.94.0
>            Reporter: Maryann Xue
>            Assignee: Maryann Xue
>
> When RS updates an unassigned ZK node to RS_ZK_REGION_OPENED, it will most probably proceed
to update the region location in META. This would cause inconsistency between the region's
location in HMaster and that in META. Not deleting this ZK node would also make further region
transitions fail with ZK exception "node already exists".
> So the master should either abort or fix this inconsistency.
> {code}
>         case RS_ZK_REGION_OPENED:
>           hri = checkIfInFailover(regionState, encodedName, regionName);
>           if (hri != null) {
>             regionState = new RegionState(hri, RegionState.State.OPEN, createTime, sn);
>             regionsInTransition.put(encodedName, regionState);
>             new OpenedRegionHandler(master, this, regionState.getRegion(), sn, expectedVersion).process();
>             failoverProcessedRegions.put(encodedName, hri);
>             break;
>           }
>           // Should see OPENED after OPENING but possible after PENDING_OPEN
>           if (regionState == null ||
>               (!regionState.isPendingOpen() && !regionState.isOpening())) {
>             LOG.warn("Received OPENED for region " +
>                 prettyPrintedRegionName +
>                 " from server " + sn + " but region was in " +
>                 " the state " + regionState + " and not " +
>                 "in expected PENDING_OPEN or OPENING states");
>             return;
>           }
>           // Handle OPENED by removing from transition and deleted zk node
>           regionState.update(RegionState.State.OPEN, createTime, sn);
>           this.executorService.submit(
>             new OpenedRegionHandler(master, this, regionState.getRegion(), sn, expectedVersion));
>           break;
> {code}
> Error logs:
> {code}
> 2012-06-29 07:07:41,149 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=swbss-hadoop-164,60020,1340888346294, region=b713fd655fa02395496c5a6e39ddf568
> 2012-06-29 07:07:41,150 WARN org.apache.hadoop.hbase.master.AssignmentManager: Received
OPENING for region b713fd655fa02395496c5a6e39ddf568 from server swbss-hadoop-164,60020,1340888346294
but region was in  the state null and not in expected PENDING_OPEN or OPENING states
> 2012-06-29 07:07:41,296 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=swbss-hadoop-164,60020,1340888346294, region=b713fd655fa02395496c5a6e39ddf568
> 2012-06-29 07:07:41,296 WARN org.apache.hadoop.hbase.master.AssignmentManager: Received
OPENING for region b713fd655fa02395496c5a6e39ddf568 from server swbss-hadoop-164,60020,1340888346294
but region was in  the state null and not in expected PENDING_OPEN or OPENING states
> 2012-06-29 07:07:41,302 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENED, server=swbss-hadoop-164,60020,1340888346294, region=b713fd655fa02395496c5a6e39ddf568
> 2012-06-29 07:07:41,302 WARN org.apache.hadoop.hbase.master.AssignmentManager: Received
OPENED for region b713fd655fa02395496c5a6e39ddf568 from server swbss-hadoop-164,60020,1340888346294
but region was in  the state null and not in expected PENDING_OPEN or OPENING states
> 2012-06-29 07:08:38,872 INFO org.apache.hadoop.hbase.master.HMaster: balance hri=CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.,
src=swbss-hadoop-006,60020,1340890678078, dest=swbss-hadoop-008,60020,1340891085175
> 2012-06-29 07:08:38,872 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting
unassignment of region CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
(offlining)
> 2012-06-29 07:08:47,875 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Sent
CLOSE to serverName=swbss-hadoop-006,60020,1340890678078, load=(requests=0, regions=0, usedHeap=0,
maxHeap=0) for region CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
> 2012-06-29 08:04:37,681 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
state=PENDING_CLOSE, ts=1340926468331, server=null
> 2012-06-29 08:04:37,681 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been PENDING_CLOSE for too long, running forced unassign again on region=CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
> 2012-06-29 08:04:47,681 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
state=PENDING_CLOSE, ts=1340926468331, server=null
> 2012-06-29 08:04:47,682 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been PENDING_CLOSE for too long, running forced unassign again on region=CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
> 2012-06-29 08:04:57,681 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
state=PENDING_CLOSE, ts=1340926468331, server=null
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message