hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-3019) Make bulk assignment on cluster startup run faster
Date Thu, 30 Sep 2010 06:20:34 GMT

     [ https://issues.apache.org/jira/browse/HBASE-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-3019:

    Attachment: bulk-v4.txt

Patch that adds a bulk open region to the regionserver and that then has the assignment manager
do bulk operations per server.

Currently, this patch does not make assignments faster than what we currently have.  Talking
about 3 minutes to assign 2k regions across 9 servers currenlty vs 4 to 5 minutes for this

Patch has concurrency issue and will play some more with it but seems like zk is bottleneck
-- all the state changes that happen for a region assignment.

> Make bulk assignment on cluster startup run faster
> --------------------------------------------------
>                 Key: HBASE-3019
>                 URL: https://issues.apache.org/jira/browse/HBASE-3019
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: stack
>         Attachments: bulk-v4.txt
> Currently, as of HBASE-3018, we come up with a bulk assignment plan that is sorted by
server.  We then spawn a thread to assign out the regions per server so we are assigning in
parallel.  This works but is still slow enough (It looks to be slower than the old assignment
where we'd do lumps of N regions at a time).  We should be able to pass a regionserver all
the regions to open in one RPC.  We need to figure how to keep up zk state while regionserver
is processing a big lot of regions.  This looks a little awkward to do since currently open
handler just opens region -- there is no notion of doing a ping while waiting to run.
> Being able to start the cluster fast is important for those times we take it down to
do major upgrade; the longer it takes to spin up, the longer our 'downtime'.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message