hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
Date Wed, 25 Jul 2018 14:46:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555781#comment-16555781
] 

Hive QA commented on HIVE-20210:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12933035/HIVE-20210.4.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14687 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12853/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12853/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12853/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12933035 - PreCommit-HIVE-Build

> Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and
conversion is minimal
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-20210
>                 URL: https://issues.apache.org/jira/browse/HIVE-20210
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Planning
>    Affects Versions: 2.3.0, 2.3.1, 2.3.2
>            Reporter: Jeffrey(Xilang) Yan
>            Assignee: Jeffrey(Xilang) Yan
>            Priority: Major
>         Attachments: HIVE-20210.2.patch, HIVE-20210.3.patch, HIVE-20210.4.patch, HIVE-20210.patch
>
>
> When conversion is minimal, simple fetch can be used only when filter on partition column
or no filter at all. But it lead to simple fetch even if filter on non-partition column. Unit
test " select * from srcpart where key > 100 limit 10 " in the nonmr_fetch.q demonstration
this issue – the unit test is not correct indeed(it should be Map Reduce  but in test it
is Simple Fetch).
> This issue lead to a serious problem when data size is huge. When conversion is more
and filter on both partition column and non-partition column, it will not chech hive.fetch.task.conversion.threshold,
which lead to the query to takes hours to finish. This issue doesn't exist in 1.2.1, how it
works should be a magic...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message