hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10535) LLAP: Cleanup map join cache when a query completes
Date Tue, 14 Jul 2015 16:52:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626637#comment-14626637
] 

Siddharth Seth commented on HIVE-10535:
---------------------------------------

- {code} ObjectCacheFactory.removeLlapQueryCache(queryId == null ? savedQueryId : queryId);
{code}
This could end up being null if the query completes before a fragment starts executing. (possible
when fragments are queued and the query is killed for example). Should either not be invoked
or removeLlapQueryCache would need to check this (ConcurrentHashMap doesn't handle nulls).

- {code} return getLlapObjectCache(queryId);{code}
Does this need to be wrapped in an ObjectCacheWrapper, to prefix the queryId correctly to
the key ?

- removeLlapQueryCache - Minor: if (isDebugEnabled) around the debug statement

- makeKey in ObjectCacheWrapper. Does this end up changing the key from queryId__HASH_MAP
to queryId_HASH_MAP. Not sure if this is significant

Changing ExecutionContext is a little strange, but I see why it's required - since the queryId
will only be available after parsing the payload. We'll likely be able to avoid this later
once LLAP execution is a little more specialized, and parses the payloads up front.

Rest looks good to me. Would prefer if [~vikram.dixit] or [~hagleitn] took a quick look at
the changes to the cache and keys though.


> LLAP: Cleanup map join cache when a query completes
> ---------------------------------------------------
>
>                 Key: HIVE-10535
>                 URL: https://issues.apache.org/jira/browse/HIVE-10535
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Sergey Shelukhin
>             Fix For: llap
>
>         Attachments: HIVE-10533.01.patch, HIVE-10533.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message