hive-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ser...@apache.org
Subject hive git commit: HIVE-12048 : metastore file metadata cache should not be used when deltas are present (Sergey Shelukhin, reviewed by Prasanth Jayachandran)
Date Tue, 13 Oct 2015 01:33:50 GMT
Repository: hive
Updated Branches:
  refs/heads/master 23ede5dd1 -> 7524ad200


HIVE-12048 : metastore file metadata cache should not be used when deltas are present (Sergey
Shelukhin, reviewed by Prasanth Jayachandran)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/7524ad20
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/7524ad20
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/7524ad20

Branch: refs/heads/master
Commit: 7524ad20054d32e84cb602c8b234a346dbf246ad
Parents: 23ede5d
Author: Sergey Shelukhin <sershe@apache.org>
Authored: Mon Oct 12 18:31:49 2015 -0700
Committer: Sergey Shelukhin <sershe@apache.org>
Committed: Mon Oct 12 18:31:49 2015 -0700

----------------------------------------------------------------------
 .../org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java     | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hive/blob/7524ad20/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java b/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
index 6ffec30..690f8dd 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
@@ -626,8 +626,12 @@ public class OrcInputFormat implements InputFormat<NullWritable, OrcStruct>,
       //       huge metastore call result that cannot be handled with in-API batching. To
have an
       //       optimal number of metastore calls, we should wait for batch-size number of
files (a
       //       few hundreds) to become available, then call metastore.
-      if (context.cacheStripeDetails) {
-        FileInfo[] infos = Context.footerCache.getAndValidate(files);
+
+      // Force local cache if we have deltas.
+      FooterCache cache = context.cacheStripeDetails ?
+          (deltas == null ? Context.footerCache : Context.localCache) : null;
+      if (cache != null) {
+        FileInfo[] infos = cache.getAndValidate(files);
         for (int i = 0; i < files.size(); ++i) {
           FileInfo info = infos[i];
           if (info != null) {


Mime
View raw message