hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6873) DISTINCT clause in aggregates is handled incorrectly by vectorized execution
Date Thu, 10 Apr 2014 06:49:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13965068#comment-13965068
] 

Jitendra Nath Pandey commented on HIVE-6873:
--------------------------------------------

It seems to me that all aggregates are working fine. The reason is that in a query like:
select sum(distinct a) from T group by b;
The map-side aggregation treats (b, a) as the key, therefore distinct values of a are getting
propagated to the reducer, and for the distinct scenario reducer only takes the key and discards
the values. 


> DISTINCT clause in aggregates is handled incorrectly by vectorized execution
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-6873
>                 URL: https://issues.apache.org/jira/browse/HIVE-6873
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.13.0, 0.14.0
>            Reporter: Remus Rusanu
>            Assignee: Remus Rusanu
>         Attachments: HIVE-6873.1.patch, HIVE-6873.2.patch
>
>
> The vectorized aggregates ignore the DISTINCT clause. This cause incorrect results. Due
to how GroupByOperatorDesc adds the DISTINCT keys to the overall aggregate keys the vectorized
aggregates do account for the extra key, but they do not process the data correctly for the
key. the reduce side the aggregates the input from the vectorized map side to results that
are only sometimes correct but mostly incorrect. HIVE-4607 tracks the proper fix, but meantime
I'm filing a bug to disable vectorized execution if DISTINCT is present. Fix is trivial.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message