kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shaofeng SHI (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3084) File not found Exception when processing union-all in TEZ mode
Date Wed, 21 Mar 2018 02:45:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407372#comment-16407372
] 

Shaofeng SHI commented on KYLIN-3084:
-------------------------------------

Hi Le Anh Vu, this change will be reverted, and a fixpack 2.3.1 will be released soon.

> File not found Exception when processing union-all in TEZ mode
> --------------------------------------------------------------
>
>                 Key: KYLIN-3084
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3084
>             Project: Kylin
>          Issue Type: Bug
>            Reporter: Wang Cheng
>            Assignee: Wang Cheng
>            Priority: Minor
>             Fix For: v2.3.0
>
>
> If hive.execution.engine=TEZ and hql contains union all, it causes exception like: file
not found when materializing the view or redistributing flat hive table.
> Here is the reason:
> http://grokbase.com/t/hive/user/162r80a2g9/anyway-to-avoid-creating-subdirectories-by-insert-with-union
> i.e. "The Tez execution of UNION is entirely parallel &
> the task-ids overlaps - so the files created have to have unique names.
> But the total counts for "Map 1" and "Map 2" are only available as the job
> runs, so they write to different dirs."
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message