hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-4139) [Hive] multi group by statement is not optimized
Date Tue, 09 Sep 2008 23:47:44 GMT
[Hive] multi group by statement is not optimized
------------------------------------------------

                 Key: HADOOP-4139
                 URL: https://issues.apache.org/jira/browse/HADOOP-4139
             Project: Hadoop Core
          Issue Type: Bug
            Reporter: Namit Jain
            Assignee: Namit Jain


A simple multi-group by statement is not optimized. A simple statement like:

FROM SRC
INSERT OVERWRITE TABLE DEST1 SELECT SRC.key, count(distinct  SUBSTR(SRC.value,4)) GROUP BY
SRC.key
INSERT OVERWRITE TABLE DEST2 SELECT SRC.key, count(distinct  SUBSTR(SRC.value,4)) GROUP BY
SRC.key;


results in making 2 copies of the data (SRC). Instead, the data can be first partially aggregated
on the distinct value and then aggregated. 
The first step can be common to all group bys.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message