Return-Path: X-Original-To: apmail-drill-issues-archive@minotaur.apache.org Delivered-To: apmail-drill-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 76CCD18B4F for ; Tue, 22 Dec 2015 16:53:49 +0000 (UTC) Received: (qmail 6888 invoked by uid 500); 22 Dec 2015 16:53:47 -0000 Delivered-To: apmail-drill-issues-archive@drill.apache.org Received: (qmail 6797 invoked by uid 500); 22 Dec 2015 16:53:47 -0000 Mailing-List: contact issues-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list issues@drill.apache.org Received: (qmail 6713 invoked by uid 99); 22 Dec 2015 16:53:47 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Dec 2015 16:53:47 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 369E12C1F60 for ; Tue, 22 Dec 2015 16:53:47 +0000 (UTC) Date: Tue, 22 Dec 2015 16:53:47 +0000 (UTC) From: "Zelaine Fong (JIRA)" To: issues@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (DRILL-4161) Make Hive Metastore client caching user configurable. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068378#comment-15068378 ] Zelaine Fong edited comment on DRILL-4161 at 12/22/15 4:53 PM: --------------------------------------------------------------- The following example shows how to change caching policy to 600 seconds after every access. Here's an example showing the parameters that need to be changed in the Hive storage plugin configuration to increase the cache invalidation time to 600 seconds, after the last time a cached object was accessed.


"configProps": {
    "hive.metastore.uris": "thrift://hive.ip.address.com:9083",
    ... // other setting, 
 
    "hive.metastore.cache-ttl-seconds": "600",
    "hive.metastore.cache-expire-after": "access"
  }

was (Author: zfong): The following example shows how to change caching policy to 600 seconds after every access. Here's an example showing the parameters that need to be changed in the Hive storage plugin configuration to increase the cache invalidation time to 600 seconds, after the last time a cached object was accessed. "configProps": { "hive.metastore.uris": "thrift://hive.ip.address.com:9083", ... // other setting, "hive.metastore.cache-ttl-seconds": "600", "hive.metastore.cache-expire-after": "access" } > Make Hive Metastore client caching user configurable. > ----------------------------------------------------- > > Key: DRILL-4161 > URL: https://issues.apache.org/jira/browse/DRILL-4161 > Project: Apache Drill > Issue Type: Bug > Reporter: Jinfeng Ni > Fix For: 1.5.0 > > > Drill leverages LoadingCache in hive metastore client, in order to avoid the long access time to hive metastore server. However, there is a tradeoff between caching stale data and the possibility of cache hit. > For instance, DRILL-3893 changes cache invalidation policy to "1 minute after last write", to avoid the chances of hitting stale data. However, it also implies that the cached data would be only valid for 1 minute after loading/write. > It's desirable to allow user to configure the caching policy, per their individual use case requirement. In particular, we probably should allow user to specify: > 1) caching invalidation policy : expire after last access, or expire after last write. > 2) cache TTL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)