Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DE0DC102AE for ; Sun, 2 Nov 2014 23:41:34 +0000 (UTC) Received: (qmail 1216 invoked by uid 500); 2 Nov 2014 23:41:34 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 1158 invoked by uid 500); 2 Nov 2014 23:41:34 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 1147 invoked by uid 99); 2 Nov 2014 23:41:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 02 Nov 2014 23:41:34 +0000 Date: Sun, 2 Nov 2014 23:41:34 +0000 (UTC) From: "Andrew Purtell (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-12219) Cache more efficiently getAll() and get() in FSTableDescriptors MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-12219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194092#comment-14194092 ] Andrew Purtell commented on HBASE-12219: ---------------------------------------- The timeout setting above is a nice tip. When hunting zombies locally you can also set the first part and second part fork modes to "always" so each test runs in its own VM, then loop the unit test suite, watch for stragglers, then jstack. Works well because you can be sure every stack in the dump is relevant for the hung test. > Cache more efficiently getAll() and get() in FSTableDescriptors > --------------------------------------------------------------- > > Key: HBASE-12219 > URL: https://issues.apache.org/jira/browse/HBASE-12219 > Project: HBase > Issue Type: Bug > Components: master > Affects Versions: 0.94.24, 0.99.1, 0.98.6.1 > Reporter: Esteban Gutierrez > Assignee: Esteban Gutierrez > Labels: scalability > Fix For: 2.0.0, 0.98.8, 0.99.2 > > Attachments: HBASE-12219-0.98.patch, HBASE-12219-0.98.v1.patch, HBASE-12219-0.99.addendum.patch, HBASE-12219-0.99.patch, HBASE-12219-v1.patch, HBASE-12219-v1.patch, HBASE-12219.v0.txt, HBASE-12219.v2.patch, HBASE-12219.v3.patch, list.png > > > Currently table descriptors and tables are cached once they are accessed for the first time. Next calls to the master only require a trip to HDFS to lookup the modified time in order to reload the table descriptors if modified. However in clusters with a large number of tables or concurrent clients and this can be too aggressive to HDFS and the master causing contention to process other requests. A simple solution is to have a TTL based cached for FSTableDescriptors#getAll() and FSTableDescriptors#TableDescriptorAndModtime() that can allow the master to process those calls faster without causing contention without having to perform a trip to HDFS for every call. to listtables() or getTableDescriptor() -- This message was sent by Atlassian JIRA (v6.3.4#6332)