hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xufeng (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5615) the master never do balance becauseof balance the parent region
Date Thu, 22 Mar 2012 07:52:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235426#comment-13235426
] 

xufeng commented on HBASE-5615:
-------------------------------

In my cluster I found this issue.

1.the balance never be executed because:
{noformat}
[2012-03-21 14:11:47,226] [DEBUG] [158-1-131-48:20000-BalancerChore] [org.apache.hadoop.hbase.master.HMaster
824] Not running balancer because 4 region(s) in transition: {3139250177b9c55fbce6856e2595b272=hbaseTable3,06640#000149,1332230348477.3139250177b9c55fbce6856e2595b272.
state=PENDING_CLOSE, ts=1332339058374, 3d7698062c1ffaa288ffa4b0630205dd=hbaseTable,12284#000051,1332214163915.3d7698062c1ffaa288ffa4b0630205dd.
st...
{noformat}

2.choose the 3139250177b9c55fbce6856e2595b272 as a sample to track.
I found it has be splited:
{noformat}
[2012-03-20 23:40:36,496] [INFO ] [regionserver20020.compactor] [org.apache.hadoop.hbase.regionserver.HRegion
563] Closed hbaseTable3,06640#000149,1332230348477.3139250177b9c55fbce6856e2595b272.
[2012-03-20 23:40:38,469] [INFO ] [regionserver20020.compactor] [org.apache.hadoop.hbase.catalog.MetaEditor
85] Offlined parent region hbaseTable3,06640#000149,1332230348477.3139250177b9c55fbce6856e2595b272.
in META
[2012-03-20 23:40:39,755] [INFO ] [regionserver20020.compactor] [org.apache.hadoop.hbase.regionserver.CompactSplitThread
181] Region split, META updated, and report to master. Parent=hbaseTable3,06640#000149,1332230348477.3139250177b9c55fbce6856e2595b272.,
new regions: hbaseTable3,06640#000149,1332286834610.bf8baeae598db2a1e87dbd0a234d1539., hbaseTable3,06723#000707,1332286834610.64ccaffa46be50a5dbc41540006afcb6..
Split took 5sec
{noformat}

3.then the backup master active one, in finishInitialization() logs,I found those logs:
[2012-03-21 11:41:46,692] [DEBUG] [master-158-1-131-48:20000] [org.apache.hadoop.hbase.master.handler.ServerShutdownHandler
348] Daughter hbaseTable3,06640#000149,1332286834610.bf8baeae598db2a1e87dbd0a234d1539. present

4.so I ensure that the parent region(3139250177b9c55fbce6856e2595b272) also in META table.

5.if 3139250177b9c55fbce6856e2595b272 in META, it will be added to AssignmentManager#regions
and AssignmentManager#servers when master rebuild the user regions.

6.balance will reference to AssignmentManager#servers to let the 3139250177b9c55fbce6856e2595b272
to move:
{noformat}
[2012-03-21 11:46:47,699] [INFO ] [158-1-131-48:20000-BalancerChore] [org.apache.hadoop.hbase.master.HMaster
849] balance hri=hbaseTable3,06640#000149,1332230348477.3139250177b9c55fbce6856e2595b272.,
src=158-1-131-48,20020,1331918756600, dest=158-1-130-11,20020,1331918756573
{noformat}

7.the parent will in RIT forever as PENDING_CLOSE state,thus balance will never be executed
{noformat}
[2012-03-21 13:13:57,201] [WARN ] [PRI IPC Server handler 3 on 20020] [org.apache.hadoop.hbase.regionserver.HRegionServer
2211] Received close for region we are not serving; 3139250177b9c55fbce6856e2595b272
{noformat}

{noformat}
[2012-03-21 11:55:55,638] [INFO ] [158-1-131-48:20000.timeoutMonitor] [org.apache.hadoop.hbase.master.AssignmentManager
2327] Regions in transition timed out:  hbaseTable3,06640#000149,1332230348477.3139250177b9c55fbce6856e2595b272.
state=PENDING_CLOSE, ts=1332330775586
[2012-03-21 11:55:55,639] [INFO ] [158-1-131-48:20000.timeoutMonitor] [org.apache.hadoop.hbase.master.AssignmentManager
2363] Region has been PENDING_CLOSE for too long, running forced unassign again on region=hbaseTable3,06640#000149,1332230348477.3139250177b9c55fbce6856e2595b272.
{noformat}
                
> the master never do balance becauseof  balance the parent region
> ----------------------------------------------------------------
>
>                 Key: HBASE-5615
>                 URL: https://issues.apache.org/jira/browse/HBASE-5615
>             Project: HBase
>          Issue Type: Bug
>            Reporter: xufeng
>            Assignee: xufeng
>            Priority: Critical
>
> the master never do balance becauseof when master do rebuildUserRegions()´╝îit will add
the parent region into  AssignmentManager#servers,
> if balancer let the parent region to move,the parent will in RIT forever.thus balance
will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message