hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6883) Dynamic partitioning optimization does not honor sort order or order by
Date Thu, 10 Apr 2014 23:37:15 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966003#comment-13966003
] 

Hive QA commented on HIVE-6883:
-------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639645/HIVE-6883.2.patch

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5571 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_dyn_part
{noformat}

Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2209/testReport
Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2209/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12639645

> Dynamic partitioning optimization does not honor sort order or order by
> -----------------------------------------------------------------------
>
>                 Key: HIVE-6883
>                 URL: https://issues.apache.org/jira/browse/HIVE-6883
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.13.0
>            Reporter: Prasanth J
>            Assignee: Prasanth J
>            Priority: Critical
>         Attachments: HIVE-6883.1.patch, HIVE-6883.2.patch
>
>
> HIVE-6455 patch does not honor sort order of the output table or order by of select statement.
The reason for the former is numDistributionKey in ReduceSinkDesc is set wrongly. It doesn't
take into account the sort columns, because of this RSOp sets the sort columns to null in
Key. Since nulls are set in place of sort columns in Key, the sort columns in Value are not
sorted. 
> The other issue is ORDER BY columns are not honored during insertion. For example
> {code}
> insert overwrite table over1k_part_orc partition(ds="foo", t) select si,i,b,f,t from
over1k_orc where t is null or t=27 order by si;
> {code}
> the select query performs order by on column 'si' in the first MR job. The following
MR job (inserted by HIVE-6455), sorts the input data on dynamic partition column 't' without
taking into account the already sorted 'si' column. This results in out of order insertion
for 'si' column.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message