pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4120) Broadcast the index file in case of POMergeCoGroup and POMergeJoin
Date Tue, 28 Apr 2015 22:57:06 GMT

     [ https://issues.apache.org/jira/browse/PIG-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai updated PIG-4120:
----------------------------
    Fix Version/s:     (was: 0.15.0)
                   0.16.0

> Broadcast the index file in case of POMergeCoGroup and POMergeJoin
> ------------------------------------------------------------------
>
>                 Key: PIG-4120
>                 URL: https://issues.apache.org/jira/browse/PIG-4120
>             Project: Pig
>          Issue Type: Sub-task
>          Components: tez
>            Reporter: Rohini Palaniswamy
>             Fix For: 0.16.0
>
>
> Currently merge join and merge cogroup use two DAGs - the first DAG creates the index
file in hdfs and second DAG does the merge join.  Similar to replicate join, we can broadcast
the index file and cache it and use it in merge join and merge cogroup. This will give better
performance and also eliminate need for the second DAG.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message