Return-Path: X-Original-To: apmail-hive-issues-archive@minotaur.apache.org Delivered-To: apmail-hive-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 83BA01896D for ; Tue, 14 Jul 2015 16:52:05 +0000 (UTC) Received: (qmail 41338 invoked by uid 500); 14 Jul 2015 16:52:05 -0000 Delivered-To: apmail-hive-issues-archive@hive.apache.org Received: (qmail 41225 invoked by uid 500); 14 Jul 2015 16:52:05 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 41109 invoked by uid 99); 14 Jul 2015 16:52:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Jul 2015 16:52:05 +0000 Date: Tue, 14 Jul 2015 16:52:05 +0000 (UTC) From: "Siddharth Seth (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-10535) LLAP: Cleanup map join cache when a query completes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-10535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626637#comment-14626637 ] Siddharth Seth commented on HIVE-10535: --------------------------------------- - {code} ObjectCacheFactory.removeLlapQueryCache(queryId == null ? savedQueryId : queryId); {code} This could end up being null if the query completes before a fragment starts executing. (possible when fragments are queued and the query is killed for example). Should either not be invoked or removeLlapQueryCache would need to check this (ConcurrentHashMap doesn't handle nulls). - {code} return getLlapObjectCache(queryId);{code} Does this need to be wrapped in an ObjectCacheWrapper, to prefix the queryId correctly to the key ? - removeLlapQueryCache - Minor: if (isDebugEnabled) around the debug statement - makeKey in ObjectCacheWrapper. Does this end up changing the key from queryId__HASH_MAP to queryId_HASH_MAP. Not sure if this is significant Changing ExecutionContext is a little strange, but I see why it's required - since the queryId will only be available after parsing the payload. We'll likely be able to avoid this later once LLAP execution is a little more specialized, and parses the payloads up front. Rest looks good to me. Would prefer if [~vikram.dixit] or [~hagleitn] took a quick look at the changes to the cache and keys though. > LLAP: Cleanup map join cache when a query completes > --------------------------------------------------- > > Key: HIVE-10535 > URL: https://issues.apache.org/jira/browse/HIVE-10535 > Project: Hive > Issue Type: Sub-task > Reporter: Siddharth Seth > Assignee: Sergey Shelukhin > Fix For: llap > > Attachments: HIVE-10533.01.patch, HIVE-10533.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)