Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 24 Oct 2012 12:26:13 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <1724354235.21374.1351081573493.JavaMail.jiratomcat@arcas>
In-Reply-To: <700571714.3161.1350682571769.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (HBASE-7018) Fix and Improve TableDescriptor
 caching for bulk assignment
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-7018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483172#comment-13483172 ] 

Hudson commented on HBASE-7018:
-------------------------------

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #233 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/233/])
    HBASE-7018 Fix and Improve TableDescriptor caching for bulk assignment (Revision 1401525)

     Result = FAILURE
gchanan : 
Files : 
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java

                
> Fix and Improve TableDescriptor caching for bulk assignment
> -----------------------------------------------------------
>
>                 Key: HBASE-7018
>                 URL: https://issues.apache.org/jira/browse/HBASE-7018
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>            Reporter: Gregory Chanan
>            Assignee: Gregory Chanan
>             Fix For: 0.94.3, 0.96.0
>
>         Attachments: 7018-trunk.v2, HBASE-7018-94.patch, HBASE-7018-94-v2.patch, HBASE-7018-94-v3.patch, HBASE-7018-trunk.patch, HBASE-7018-v3-trunk.patch, HBASE-7018-v4-trunk.patch
>
>
> HBASE-6214 backported HBASE-5998 (Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receiving an open regions request), but it's buggy on 0.94 (0.96 appears correct):
> {code}
>     HTableDescriptor htd = null;
>     if (htds == null) {
>       htd = this.tableDescriptors.get(region.getTableName());
>     } else {
>       htd = htds.get(region.getTableNameAsString());
>       if (htd == null) {
>         htd = this.tableDescriptors.get(region.getTableName());
>         htds.put(region.getRegionNameAsString(), htd);
>       }
>     }
> {code}
> i.e. we get the tableName from the map but write the regionName.
> Even fixing this, it looks like there are areas for improvement:
> 1) FSTableDescriptors already has a cache (though it goes to the NameNode each time through to check we have the latest copy.  May as well combine these two caches, might be a performance win as well since we don't need to write to multiple caches.
> 2) FSTableDescriptors makes two RPCs to the NameNode when it encounters a new table.  So the total number of RPCs necessary for a bulk assign (without caching is):
> #regions + #tables
> (with caching):
> min(#regions,#tables) + #tables = #tables + #tables = 2 * #tables
> We can make this only one RPC, yielding:
> #tables
> Probably not a big deal for most users, but in a multi-tenant situation where the number of regions being bulk assigned approaches the number of tables being bulk assigned, this could be a nice performance win.
> Benchmarks coming.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira