pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-4805) Auto-parallelism estimation should use split sub plan specific to the successor
Date Wed, 10 Feb 2016 20:22:18 GMT
Rohini Palaniswamy created PIG-4805:
---------------------------------------

             Summary: Auto-parallelism estimation should use split sub plan specific to the
successor
                 Key: PIG-4805
                 URL: https://issues.apache.org/jira/browse/PIG-4805
             Project: Pig
          Issue Type: Improvement
            Reporter: Rohini Palaniswamy


In PIG-4802, we determine different parallelism factors for different successors (edges).
For eg: If we have two successors, one with combine plan and other without
we want to compute lesser parallelism factor for the one with the combine plan
as that edge will get less data.

To be more perfect, we need only look at the split sub-plan that
writes to that successor edge. If there is a FILTER in one sub-plan it is accounted
for all the successors now which is not right.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message