hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4010) HMaster.createTable could be heavily optimized
Date Tue, 21 Jun 2011 21:41:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052861#comment-13052861

Ted Yu commented on HBASE-4010:

For item 1, I think you meant createHRegion(). Some variables are computed before HRegion.newHRegion()
is called. So I didn't refactor the method.
I can refactor addRegionsToMeta() and addRegionToMeta()

For batch Put, shall we choose a larger batch size (e.g. 100). Some users would create table
with many regions (e.g. 70k). But I don't have strong opinion on this item.

I am running unit tests.
TestMasterFailover hung in the suite. When I ran it manually, it passed.
I started another round of tests.

> HMaster.createTable could be heavily optimized
> ----------------------------------------------
>                 Key: HBASE-4010
>                 URL: https://issues.apache.org/jira/browse/HBASE-4010
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.3
>            Reporter: Jean-Daniel Cryans
>            Assignee: Ted Yu
>             Fix For: 0.92.0
>         Attachments: 4010-v2.txt, 4010.txt
> Looking at the createTable method in HMaster (the one that's private), we seem to be
very inefficient:
>  - We set the enabled flag for the table for every region (should be done only once).
>  - Every time we create a new region we create a new HLog and then close it (reuse one
instead or see if it's really necessary).
>  - We do one RPC to .META. per region (we should batch put).
> This should provide drastic speedups even for those creating tables with just 50 regions.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message