Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Fri, 13 May 2016 22:19:12 +0000 (UTC)
From: "Lefty Leverenz (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.12969495.1463131222000.188367.1463177952853@Atlassian.JIRA>
In-Reply-To: <JIRA.12969495.1463131222000@Atlassian.JIRA>
References: <JIRA.12969495.1463131222000@Atlassian.JIRA> <JIRA.12969495.1463131222507@arcas>
Subject: [jira] [Commented] (HIVE-13756) Map failure attempts to delete
 reducer _temporary directory on multi-query pig query
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 13 May 2016 22:19:14 -0000


    [ https://issues.apache.org/jira/browse/HIVE-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283231#comment-15283231 ] 

Lefty Leverenz commented on HIVE-13756:
---------------------------------------

[~cdrome], please specify Target Version instead of Fix Version -- you can make the change with the Edit button (top left).

Fix Version is only supposed to be set _after a patch gets committed_ so that the release notes will only pick up issues that are already patched in the code.

* [JIRA Guidelines | https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-JIRAGuidelines]

Thanks.

> Map failure attempts to delete reducer _temporary directory on multi-query pig query
> ------------------------------------------------------------------------------------
>
>                 Key: HIVE-13756
>                 URL: https://issues.apache.org/jira/browse/HIVE-13756
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 1.2.1, 2.0.0
>            Reporter: Chris Drome
>            Assignee: Chris Drome
>             Fix For: 1.2.1, 2.0.0
>
>         Attachments: HIVE-13756-branch-1.patch, HIVE-13756.patch
>
>
> A pig script, executed with multi-query enabled, that reads the source data and writes it as-is into TABLE_A as well as performing a group-by operation on the data which is written into TABLE_B can produce erroneous results if any map fails. This results in a single MR job that writes the map output to a scratch directory relative to TABLE_A and the reducer output to a scratch directory relative to TABLE_B.
> If one or more maps fail it will delete the attempt data relative to TABLE_A, but it also deletes the _temporary directory relative to TABLE_B. This has the unintended side-effect of preventing subsequent maps from committing their data. This means that any maps which successfully completed before the first map failure will have its data committed as expected, other maps not, resulting in an incomplete result set.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)