hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-23286) The clean-up in case of an aborted FileSinkOperator is not correct for ACID direct insert
Date Fri, 24 Apr 2020 23:52:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-23286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091978#comment-17091978
] 

Hive QA commented on HIVE-23286:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001046/HIVE-23286.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17141 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/21925/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21925/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21925/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 13001046 - PreCommit-HIVE-Build

> The clean-up in case of an aborted FileSinkOperator is not correct for ACID direct insert
> -----------------------------------------------------------------------------------------
>
>                 Key: HIVE-23286
>                 URL: https://issues.apache.org/jira/browse/HIVE-23286
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Marta Kuczora
>            Assignee: Marta Kuczora
>            Priority: Major
>             Fix For: 4.0.0
>
>         Attachments: HIVE-23286.1.patch, HIVE-23286.1.patch
>
>
> In FileSinkOperator there is a code path when the operator is aborted:
> {noformat}
>     } else {
>       // Will come here if an Exception was thrown in map() or reduce().
>       // Hadoop always call close() even if an Exception was thrown in map() or
>       // reduce().
>       for (FSPaths fsp : valToPaths.values()) {
>         fsp.abortWritersAndUpdaters(fs, abort, !autoDelete && isNativeTable()
&& !conf.isMmTable());
>       }
> {noformat}
> In this part, the fsp.abortWritersAndUpdaters method call should consider the conf.isDirectInsert
parameter as well. Since this parameter is missing, this method can delete the content of
the table if an insert failure aborts the FileSinkOperator and the ACID direct insert it turned
on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message