hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-555) Only one Worker in HRS; on startup, if assigned tens of regions, havoc of reassignments because open processing is done in series
Date Tue, 01 Apr 2008 21:33:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584318#action_12584318
] 

stack commented on HBASE-555:
-----------------------------

Going to commit this thing.  Just tested it over on the Lars cluster (2100 regions on 20 servers).
 The two fixes in this patch made it so on restart, there were no more "should not have opened
region's" -- all came up smoothly and regions are spread pretty evenly (looks better than
what it used to be but I only did one restart).

> Only one Worker in HRS; on startup, if assigned tens of regions, havoc of reassignments
because open processing is done in series
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-555
>                 URL: https://issues.apache.org/jira/browse/HBASE-555
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.16.0, 0.2.0, 0.1.0
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>         Attachments: 555-0.1-v2.patch, 555-0.1.patch
>
>
> On the Lars clusters, he's up into the thousands of regions.  Starting this cluster,
there is a load of churn in the master log as we assign regions, they report their opening
and then after the hbase.hbasemaster.maxregionopen of one minute elapses, we assign the region
elsewhere.
> Problem seems to be the fact that we only run a single Worker thread in our regionserver;
means that region opens are processed in series.
> For example, the below shows when a master assigned a region and then the regionserver
side log when it got around to opening it:
> {code}
> 2008-03-29 04:48:51,638 INFO org.apache.hadoop.hbase.HMaster: assigning region pdc-docs,US20060158177_20060720,1205765009844
to server 192.168.105.19:60020
> ..
> 2008-03-29 04:50:58,124 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN :
pdc-docs,US20060158177_20060720,1205765009844
> {code}
> There is > 2 minutes between the two loggings (I checked clocks on this cluster and
they are synced).
> Looking in the regionserver log, its just filled with logging on the opening of regions.
 The region opens are running pretty fast at about a second each but there are hundreds of
regions to open in this case so its easy to go over our default of 60 seconds.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message