hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16898) Validation of source file after distcp in repl load
Date Tue, 19 Sep 2017 10:15:01 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171464#comment-16171464
] 

Hive QA commented on HIVE-16898:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887786/HIVE-16898.3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 11043 tests executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash] (batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2] (batchId=89)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=215)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=215)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testIncrementalLoadFailAndRetry (batchId=218)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=240)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=240)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=240)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeProxyAuth (batchId=240)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=240)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=240)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=240)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6880/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6880/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6880/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12887786 - PreCommit-HIVE-Build

> Validation of source file after distcp in repl load 
> ----------------------------------------------------
>
>                 Key: HIVE-16898
>                 URL: https://issues.apache.org/jira/browse/HIVE-16898
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 3.0.0
>            Reporter: anishek
>            Assignee: Daniel Dai
>             Fix For: 3.0.0
>
>         Attachments: HIVE-16898.1.patch, HIVE-16898.2.patch, HIVE-16898.3.patch
>
>
> time between deciding the source and destination path for distcp to invoking of distcp
can have a change of the source file, hence distcp might copy the wrong file to destination,
hence we should an additional check on the checksum of the source file path after distcp finishes
to make sure the path didnot change during the copy process. if it has take additional steps
to delete the previous file on destination and copy the new source and repeat the same process
as above till we copy the correct file. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message