hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6459) improve speed of create table
Date Fri, 19 Oct 2012 13:58:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480027#comment-13480027
] 

Ted Yu commented on HBASE-6459:
-------------------------------

I ran the failed test locally and it passed.
{code}
Running org.apache.hadoop.hbase.client.TestFromClientSide
2012-10-19 06:55:19.937 java[39861:1903] Unable to load realm mapping info from SCDynamicStore
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 76.821 sec
{code}
Will integrate later today if there is no more review comment.
                
> improve speed of create table
> -----------------------------
>
>                 Key: HBASE-6459
>                 URL: https://issues.apache.org/jira/browse/HBASE-6459
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.94.0
>            Reporter: zhou wenjian
>            Assignee: Zhou wenjian
>             Fix For: 0.96.0
>
>         Attachments: 6459v3.txt, 6459.v4, HBASE-6459-90.patch, HBASE-6459-92.patch, HBASE-6459-94.patch,
HBASE-6459-trunk.patch, HBASE-6459-trunk-v2.patch
>
>
> in CreateTableHandler
> for (int regionIdx = 0; regionIdx < this.newRegions.length; regionIdx++) {
>       HRegionInfo newRegion = this.newRegions[regionIdx];
>       // 1. Create HRegion
>       HRegion region = HRegion.createHRegion(newRegion,
>         this.fileSystemManager.getRootDir(), this.conf,
>         this.hTableDescriptor, null, false, true);
>       regionInfos.add(region.getRegionInfo());
>       if (regionIdx % batchSize == 0) {
>         // 2. Insert into META
>         MetaEditor.addRegionsToMeta(this.catalogTracker, regionInfos);
>         regionInfos.clear();
>       }
>       // 3. Close the new region to flush to disk.  Close log file too.
>       region.close();
>     }
> All the region will be create serially.
> If we have thousands of regions, that will be a huge cost.
> We can improve it by create the region in parallel

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message