hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang Haihua (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
Date Thu, 14 Jun 2018 05:48:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16512005#comment-16512005
] 

Wang Haihua commented on HIVE-12812:
------------------------------------

seems we have encountered this issue, as mapreduce is suffering loading data generated by
tez union all

> Enable mapred.input.dir.recursive by default to support union with aggregate function
> -------------------------------------------------------------------------------------
>
>                 Key: HIVE-12812
>                 URL: https://issues.apache.org/jira/browse/HIVE-12812
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 1.2.1, 2.1.0
>            Reporter: Chaoyu Tang
>            Assignee: Chaoyu Tang
>            Priority: Major
>         Attachments: HIVE-12812.patch, HIVE-12812.patch, HIVE-12812.patch
>
>
> When union remove optimization is enabled, union query with aggregate function writes
its subquery intermediate results to subdirs which needs mapred.input.dir.recursive to be
enabled in order to be fetched. This property is not defined by default in Hive and often
ignored by user, which causes the query failure and is hard to be debugged.
> So we need set mapred.input.dir.recursive to true whenever union remove optimization
is enabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message