hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ryan rawson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1583) Start/Stop of large cluster untenable
Date Thu, 25 Jun 2009 01:19:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723845#action_12723845

ryan rawson commented on HBASE-1583:

some thoughts:

- get rid of compactions during startup/shutdown.  There is no reason to run compactions at
those times.
- dont worry about compacting reference regions as quickly, we can wait a few minutes if necessary.
- make shut down flush memstore as fast as possible, then get the heck outta there.

we need to do more work on bringing up regions as fast as possible in 0.21.  The master needs
to blast region assignments out there in a threaded/performant manner, rather than waiting
for 3-second checkins to do their magic.

Finally I have decided to up the size of a region on my own set-up.  256MB seems a little
too small, and with even twice as large regions, i end up with less than half the region count!

> Start/Stop of large cluster untenable
> -------------------------------------
>                 Key: HBASE-1583
>                 URL: https://issues.apache.org/jira/browse/HBASE-1583
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>             Fix For: 0.20.0
> Starting and stopping a loaded large cluster is way too flakey and takes too long.  This
is 0.19.x but same issues apply to TRUNK I'd say.
> At pset with our > 100 nodes carrying 6k regions:
> + shutdown takes way too long.... maybe ten minutes or so.  We compact regions inline
with shutdown.  We should just go down.  It doesn't seem like all regionservers go down everytime
> + startup is a mess with our assigning out regions an rebalancing at same time.  By time
that the compactions on open run, it can be near an hour before whole thing settles down and
becomes useable

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message