pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-4165) Rework Hash based aggregation for Tez
Date Thu, 11 Sep 2014 01:20:34 GMT
Rohini Palaniswamy created PIG-4165:
---------------------------------------

             Summary: Rework Hash based aggregation for Tez
                 Key: PIG-4165
                 URL: https://issues.apache.org/jira/browse/PIG-4165
             Project: Pig
          Issue Type: Sub-task
            Reporter: Rohini Palaniswamy


pig.exec.mapPartAgg (https://wiki.apache.org/pig/PigHashBasedAggInMap) is currently only used
for group by if there is any aggregation to be done. In Tez, should enable hash based grouping
for normal and skewed joins, groupby with or without aggregation on the map side. This will
play well with the IFile optimizations done and support for writing key and list of values
(TEZ-1260) and reduce the sorting time on the map side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message