From commits-return-3144-archive-asf-public=cust-asf.ponee.io@impala.apache.org Mon Feb 12 22:30:58 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 41D1718067B for ; Mon, 12 Feb 2018 22:30:58 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 31DCB160C61; Mon, 12 Feb 2018 21:30:58 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 823AC160C30 for ; Mon, 12 Feb 2018 22:30:57 +0100 (CET) Received: (qmail 43986 invoked by uid 500); 12 Feb 2018 21:30:56 -0000 Mailing-List: contact commits-help@impala.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@impala.apache.org Delivered-To: mailing list commits@impala.apache.org Received: (qmail 43925 invoked by uid 99); 12 Feb 2018 21:30:56 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Feb 2018 21:30:56 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id E84C3E38B8; Mon, 12 Feb 2018 21:30:55 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: tarmstrong@apache.org To: commits@impala.apache.org Date: Mon, 12 Feb 2018 21:30:57 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [3/5] impala git commit: IMPALA-6486: Fix INVALIDATE METADATA hang after statestore restart IMPALA-6486: Fix INVALIDATE METADATA hang after statestore restart This commit fixes an issue where an INVALIDATE METADATA statement will hang if executed after a statestore restart. The issue was that the CatalogObjectVersionQueue was not properly reset when a non-delta catalog topic update was received in an impalad, causing it to record duplicate catalog versions and, hence, not properly advancing the minimum catalog object watermark. The latter is used by INVALIDATE METADATA to determine when the effects of that operation have been applied to the local impalad catalog cache. Change-Id: Icf3ee31c28884ed79c5dbc700ccd4ef761015c68 Reviewed-on: http://gerrit.cloudera.org:8080/9232 Reviewed-by: Dimitris Tsirogiannis Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/impala/repo Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/cdbacb1c Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/cdbacb1c Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/cdbacb1c Branch: refs/heads/2.x Commit: cdbacb1c510eb54c9a03b9a37ae96012f15e5629 Parents: bf89285 Author: Dimitris Tsirogiannis Authored: Tue Feb 6 15:11:53 2018 -0800 Committer: Impala Public Jenkins Committed: Fri Feb 9 17:10:16 2018 +0000 ---------------------------------------------------------------------- .../org/apache/impala/catalog/CatalogObjectVersionQueue.java | 2 ++ .../main/java/org/apache/impala/catalog/ImpaladCatalog.java | 7 +++---- 2 files changed, 5 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/impala/blob/cdbacb1c/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java ---------------------------------------------------------------------- diff --git a/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java b/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java index 5fcd398..e24e66b 100644 --- a/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java +++ b/fe/src/main/java/org/apache/impala/catalog/CatalogObjectVersionQueue.java @@ -70,4 +70,6 @@ public class CatalogObjectVersionQueue { removeVersion(catalogObject.getCatalogVersion()); } } + + public void clear() { objectVersions_.clear(); } } http://git-wip-us.apache.org/repos/asf/impala/blob/cdbacb1c/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java ---------------------------------------------------------------------- diff --git a/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java b/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java index 79960e4..99bd23e 100644 --- a/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java +++ b/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java @@ -97,13 +97,12 @@ public class ImpaladCatalog extends Catalog { // Used during table creation. private final String defaultKuduMasterHosts_; - /** - * C'tor used by tests that need to validate the ImpaladCatalog outside of the - * CatalogServer. - */ public ImpaladCatalog(String defaultKuduMasterHosts) { super(); defaultKuduMasterHosts_ = defaultKuduMasterHosts; + // Ensure the contents of the CatalogObjectVersionQueue instance are cleared when a + // new instance of ImpaladCatalog is created (see IMPALA-6486). + CatalogObjectVersionQueue.INSTANCE.clear(); } /**