hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15441) Provide a config to timeout long compiling queries
Date Fri, 16 Dec 2016 01:53:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753136#comment-15753136
] 

Sergey Shelukhin commented on HIVE-15441:
-----------------------------------------

1) Is there need to wake up every second? It could just sleep for the duration and check when
awoken to avoid spurious wake-ups. Also it's nice to honor interrupts.
2) What is the thread that it actually kills? E.g. in CLI would it kill the main thread, or
in HS2 would the handler it kills be recovered, and would the user receive a useful error
message?

> Provide a config to timeout long compiling queries
> --------------------------------------------------
>
>                 Key: HIVE-15441
>                 URL: https://issues.apache.org/jira/browse/HIVE-15441
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Planning
>            Reporter: Chao Sun
>            Assignee: Chao Sun
>         Attachments: HIVE-15441.1.patch
>
>
> Sometimes Hive users have long compiling queries which may need to scan thousands or
even more partitions (perhaps by accident). The compilation process may take a very long time,
especially in {{getInputSummary}} where it need to make NN calls to get info about each input
path.
> This is bad because it may block many other queries. Parallel compilation may be useful
but still {{getInputSummary}} has a global lock. In this case, it makes sense to provide Hive
admin with a config to put a timeout limit for compilation, so that these "bad" queries can
be blocked.
> Note https://issues.apache.org/jira/browse/HIVE-12431 also tries to address similar issue.
However it cancels those queries that are waiting for the compile lock, which I think is not
so useful for our case since the *query under compile is the one to be blamed.*



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message