hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21402) Compaction state remains 'working' when major compaction fails
Date Mon, 11 Mar 2019 14:11:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789614#comment-16789614
] 

Ashutosh Chauhan commented on HIVE-21402:
-----------------------------------------

I am unsure of how to deal with unchecked exceptions. IMHO, its not useful to catch Throwable
since in case of unchecked exception its very likely that compaction will fail in next iteration
too, likely that error will be encountered every time (e.g., was the case here of missing
jar). In such cases, its better to let Throwable escape (or raise InterrruptedException) so
that its dealt with in caller which should then fail the process. For end user its not useful
that HS2 keeps on running where every compaction fails.
On the other hand there is already catch(Throwable) in the outer loop : https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java#L238


> Compaction state remains 'working' when major compaction fails
> --------------------------------------------------------------
>
>                 Key: HIVE-21402
>                 URL: https://issues.apache.org/jira/browse/HIVE-21402
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 4.0.0
>            Reporter: Peter Vary
>            Assignee: Peter Vary
>            Priority: Major
>         Attachments: HIVE-21402.patch
>
>
> When calcite is not on the HMS classpath, and query based compaction is enabled then
the compaction fails with NoClassDefFound error. Since the catch block only catches Exceptions
the following code block is not executed:
> {code:java}
> } catch (Exception e) {
>   LOG.error("Caught exception while trying to compact " + ci +
>       ".  Marking failed to avoid repeated failures, " + StringUtils.stringifyException(e));
>   msc.markFailed(CompactionInfo.compactionInfoToStruct(ci));
>   msc.abortTxns(Collections.singletonList(compactorTxnId));
> }
> {code}
> So the compaction is not set to failed.
> Would be better to catch Throwable instead of Exception



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message