pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aniket Mokashi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-2668) projection pushdown should work through Split
Date Wed, 25 Apr 2012 21:42:17 GMT
Aniket Mokashi created PIG-2668:
-----------------------------------

             Summary: projection pushdown should work through Split 
                 Key: PIG-2668
                 URL: https://issues.apache.org/jira/browse/PIG-2668
             Project: Pig
          Issue Type: Bug
            Reporter: Aniket Mokashi


A = load ...
B1 = filter A by cond1;
B2 = filter A by cond2;
B3 = filter A by cond3;

B = union B1, B2, B3; does not push projection.

Load (A)
|
Split
 |
---------------------------
|             |
Filter(B1)   Filter(B2) ...

Because of the split operator introduced between the filter conditions and load, the filter
does not get pushed into the load function.

A simple way to fix this in pig would be to not share the load across the filter operators.
Another option is to push the condition (B1 or B2 or B3) into Load operator and retain rest
of the current plan (split and filters following the split).

You can of course achieve the same effect by having a separate load statement as input for
each of the filters.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message