Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 3 Jan 2017 20:30:58 +0000 (UTC)
From: "Sergey Shelukhin (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13028545.1481848208000.631939.1483475458419@Atlassian.JIRA>
In-Reply-To: <JIRA.13028545.1481848208000@Atlassian.JIRA>
References: <JIRA.13028545.1481848208000@Atlassian.JIRA> <JIRA.13028545.1481848208104@arcas>
Subject: [jira] [Comment Edited] (HIVE-15441) Provide a config to timeout
 long compiling queries
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 03 Jan 2017 20:31:00 -0000


    [ https://issues.apache.org/jira/browse/HIVE-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796098#comment-15796098 ] 

Sergey Shelukhin edited comment on HIVE-15441 at 1/3/17 8:30 PM:
-----------------------------------------------------------------

I think the only way to achieve this would be to add interrupt checks on case by case basis (e.g. during partition fetching, and in some intermediate points during compilation). Explicit checks would also allow for a better error message.
Talking to remote metastore or metastore DB should be blocking, right? So at least the network call will be interrupted, and hopefully won't just lead to retry, undecipherable errors or some other such nonsense typical of the Hive exception handing :)


was (Author: sershe):
I think the only way to achieve this would be to add interrupt checks on case by case basis.
Talking to remote metastore or metastore DB should be blocking, right? So at least the network call will be interrupted, and hopefully won't just lead to retry, undecipherable errors or some other such nonsense typical of the Hive exception handing :)

> Provide a config to timeout long compiling queries
> --------------------------------------------------
>
>                 Key: HIVE-15441
>                 URL: https://issues.apache.org/jira/browse/HIVE-15441
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Planning
>            Reporter: Chao Sun
>            Assignee: Chao Sun
>         Attachments: HIVE-15441.1.patch
>
>
> Sometimes Hive users have long compiling queries which may need to scan thousands or even more partitions (perhaps by accident). The compilation process may take a very long time, especially in {{getInputSummary}} where it need to make NN calls to get info about each input path.
> This is bad because it may block many other queries. Parallel compilation may be useful but still {{getInputSummary}} has a global lock. In this case, it makes sense to provide Hive admin with a config to put a timeout limit for compilation, so that these "bad" queries can be blocked.
> Note https://issues.apache.org/jira/browse/HIVE-12431 also tries to address similar issue. However it cancels those queries that are waiting for the compile lock, which I think is not so useful for our case since the *query under compile is the one to be blamed.*


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)