impala-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tarmstr...@apache.org
Subject [3/5] impala git commit: IMPALA-6486: Fix INVALIDATE METADATA hang after statestore restart
Date Mon, 12 Feb 2018 21:30:57 GMT
IMPALA-6486: Fix INVALIDATE METADATA hang after statestore restart

This commit fixes an issue where an INVALIDATE METADATA statement will
hang if executed after a statestore restart. The issue was that the
CatalogObjectVersionQueue was not properly reset when a non-delta
catalog topic update was received in an impalad, causing it to record
duplicate catalog versions and, hence, not properly advancing the minimum
catalog object watermark. The latter is used by INVALIDATE METADATA to
determine when the effects of that operation have been applied to the
local impalad catalog cache.

Change-Id: Icf3ee31c28884ed79c5dbc700ccd4ef761015c68
Reviewed-on: http://gerrit.cloudera.org:8080/9232
Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Tested-by: Impala Public Jenkins


Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/cdbacb1c
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/cdbacb1c
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/cdbacb1c

Branch: refs/heads/2.x
Commit: cdbacb1c510eb54c9a03b9a37ae96012f15e5629
Parents: bf89285
Author: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Authored: Tue Feb 6 15:11:53 2018 -0800
Committer: Impala Public Jenkins <impala-public-jenkins@gerrit.cloudera.org>
Committed: Fri Feb 9 17:10:16 2018 +0000

----------------------------------------------------------------------
 .../org/apache/impala/catalog/CatalogObjectVersionQueue.java  | 2 ++
 .../main/java/org/apache/impala/catalog/ImpaladCatalog.java   | 7 +++----
 2 files changed, 5 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/impala/blob/cdbacb1c/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java
----------------------------------------------------------------------
diff --git a/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java b/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java
index 5fcd398..e24e66b 100644
--- a/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java
+++ b/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java
@@ -70,4 +70,6 @@ public class CatalogObjectVersionQueue {
       removeVersion(catalogObject.getCatalogVersion());
     }
   }
+
+  public void clear() { objectVersions_.clear(); }
 }

http://git-wip-us.apache.org/repos/asf/impala/blob/cdbacb1c/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
----------------------------------------------------------------------
diff --git a/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java b/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
index 79960e4..99bd23e 100644
--- a/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
+++ b/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
@@ -97,13 +97,12 @@ public class ImpaladCatalog extends Catalog {
   // Used during table creation.
   private final String defaultKuduMasterHosts_;
 
-  /**
-   * C'tor used by tests that need to validate the ImpaladCatalog outside of the
-   * CatalogServer.
-   */
   public ImpaladCatalog(String defaultKuduMasterHosts) {
     super();
     defaultKuduMasterHosts_ = defaultKuduMasterHosts;
+    // Ensure the contents of the CatalogObjectVersionQueue instance are cleared when a
+    // new instance of ImpaladCatalog is created (see IMPALA-6486).
+    CatalogObjectVersionQueue.INSTANCE.clear();
   }
 
   /**


Mime
View raw message