drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject drill git commit: edit to hive metadata caching
Date Tue, 02 Feb 2016 21:24:06 GMT
Repository: drill
Updated Branches:
  refs/heads/gh-pages 6a0417d86 -> 657633d7b


edit to hive metadata caching


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/657633d7
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/657633d7
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/657633d7

Branch: refs/heads/gh-pages
Commit: 657633d7bbdee44ed2fbd5b13339ecf91624730b
Parents: 6a0417d
Author: Bridget Bevens <bbevens@maprtech.com>
Authored: Tue Feb 2 13:21:08 2016 -0800
Committer: Bridget Bevens <bbevens@maprtech.com>
Committed: Tue Feb 2 13:21:08 2016 -0800

----------------------------------------------------------------------
 .../026-hive-metadata-caching.md                | 50 ++++++++++----------
 1 file changed, 25 insertions(+), 25 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/657633d7/_docs/performance-tuning/026-hive-metadata-caching.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/026-hive-metadata-caching.md b/_docs/performance-tuning/026-hive-metadata-caching.md
index 67781a1..efb613a 100644
--- a/_docs/performance-tuning/026-hive-metadata-caching.md
+++ b/_docs/performance-tuning/026-hive-metadata-caching.md
@@ -1,15 +1,15 @@
 ---
 title: "Hive Metadata Caching"
-date:  
+date: 2016-02-02 21:21:09 UTC
 parent: "Performance Tuning"
 ---
 
 Drill caches Hive metadata in a Hive metastore client cache that resides in Drill instead
of accessing the Hive metastore directly. During a query, Drill can access metadata faster
from the cache than from the Hive metastore. By default, the Hive metastore client cache has
a TTL (time to live) of 60 seconds. The TTL is how long cache entries exist before the cache
reloads metadata from the Hive metastore. Drill expires an entry in the cache 60 seconds after
the following events:  
 
-*  creation of the entry
-*  a read or write operation on the entry
-*  the most recent replacement of the entry value  
-
+*  creation of the entry
+*  a read or write operation on the entry
+*  the most recent replacement of the entry value  
+
 You can modify the TTL depending on how frequently the Hive metadata is updated. If the Hive
metadata is updated frequently, decrease the cache TTL value. If Hive metadata is updated
infrequently, increase the cache TTL value.
 
 For example, when you run a Drill query on a Hive table, Drill refreshes the cache 60 seconds
after the read on the table. If the table is updated in Hive within that 60 second window
and you issue another query on the table, the query would return incorrect results or an execution
error due to a stale cache. In such a scenario where Hive metadata is changing so quickly,
you may want to reduce the cache TTL to 2 seconds so that Drill refreshes the cache more frequently.
 
@@ -25,26 +25,26 @@ To configure the Hive metastore client cache in Drill, complete the following
st
 1. Start the [Drill Web Console]({{site.baseurl}}/docs/starting-the-web-console/).
 2. Select the **Storage** tab.
 3. Click **Update** next to the “hive” storage plugin.
-4. Add the following parameters:  
-
+4. Add the following parameters:  
+
               "hive.metastore.cache-ttl-seconds": "<value>",
               "hive.metastore.cache-expire-after": "<value>"  
 The `cache-ttl-seconds` value can be any non-negative value, including 0, which turns caching
off. The `cache-expire-after` value can be “`access`” or “`write`”. Access indicates
expiry after a read or write operation, and write indicates expiry after a write operation
only.
-5. **Enable** the storage plugin to save the changes.  
-
-Example:  
-
-       {
-             "type": "hive",
-             "enabled": false,
-             "configProps": {
-               "hive.metastore.uris": "",
-               "javax.jdo.option.ConnectionURL": "jdbc:derby:;databaseName=../sample-data/drill_hive_db;create=true",
-               "hive.metastore.warehouse.dir": "/tmp/drill_hive_wh",
-               "fs.default.name": "file:///",
-               "hive.metastore.sasl.enabled": "false"
-        	   "hive.metastore.cache-ttl-seconds": "2",
-               "hive.metastore.cache-expire-after": "access"
-        
-         	  }
-       	}
+5. **Enable** the storage plugin to save the changes.  
+
+Example:  
+
+       {
+             "type": "hive",
+             "enabled": true,
+             "configProps": {
+               "hive.metastore.uris": "",
+               "javax.jdo.option.ConnectionURL": "jdbc:derby:;databaseName=../sample-data/drill_hive_db;create=true",
+               "hive.metastore.warehouse.dir": "/tmp/drill_hive_wh",
+               "fs.default.name": "file:///",
+               "hive.metastore.sasl.enabled": "false"
+        	   "hive.metastore.cache-ttl-seconds": "2",
+               "hive.metastore.cache-expire-after": "access"
+        
+         	  }
+       	}


Mime
View raw message