hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13756) Map failure attempts to delete reducer _temporary directory on multi-query pig query
Date Thu, 23 Jun 2016 19:10:16 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347002#comment-15347002
] 

Hive QA commented on HIVE-13756:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12812613/HIVE-13756.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10264 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/234/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/234/console
Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-234/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12812613 - PreCommit-HIVE-MASTER-Build

> Map failure attempts to delete reducer _temporary directory on multi-query pig query
> ------------------------------------------------------------------------------------
>
>                 Key: HIVE-13756
>                 URL: https://issues.apache.org/jira/browse/HIVE-13756
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 1.2.1, 2.0.0
>            Reporter: Chris Drome
>            Assignee: Chris Drome
>         Attachments: HIVE-13756-branch-1.patch, HIVE-13756.1-branch-1.patch, HIVE-13756.1.patch,
HIVE-13756.patch
>
>
> A pig script, executed with multi-query enabled, that reads the source data and writes
it as-is into TABLE_A as well as performing a group-by operation on the data which is written
into TABLE_B can produce erroneous results if any map fails. This results in a single MR job
that writes the map output to a scratch directory relative to TABLE_A and the reducer output
to a scratch directory relative to TABLE_B.
> If one or more maps fail it will delete the attempt data relative to TABLE_A, but it
also deletes the _temporary directory relative to TABLE_B. This has the unintended side-effect
of preventing subsequent maps from committing their data. This means that any maps which successfully
completed before the first map failure will have its data committed as expected, other maps
not, resulting in an incomplete result set.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message