hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Na Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7651) Investigate why union two RDDs generated from two MapTrans does not get the right result
Date Mon, 11 Aug 2014 18:12:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093083#comment-14093083
] 

Na Yang commented on HIVE-7651:
-------------------------------

This issue was caused by a single jobConf instance are used by multiple MapTrans. The fix
is included in the patch of HIVE-7541.  

> Investigate why union two RDDs generated from two MapTrans does not get the right result
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-7651
>                 URL: https://issues.apache.org/jira/browse/HIVE-7651
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>            Reporter: Na Yang
>
> If the SparkWork has two map works as root, then use the current generate(basework) API
to generate two mapTran. union the RDDs processed by the two mapTrans does not generate the
correct result. 
> If two input RDDs come from different data tables, then the union result is empty.
> if two input RDDs come from the same data table, then the union result is not correct.
The same row of data happen 4 times in the union result.
> Need to investigate why this happen and how to fix it.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message