pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-419) Combiner optimizations extended to nested foreach statements as well
Date Wed, 20 Oct 2010 22:34:24 GMT

    [ https://issues.apache.org/jira/browse/PIG-419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923208#action_12923208

Thejas M Nair commented on PIG-419:

Adding a related item from http://wiki.apache.org/pig/PigJournal .

"Pig Scripts that have a foreach with a nested limit or filter do not use the combiner even
when they could. Not all filters can use the combiner, but in some cases they can. I think
all limits could at least apply the limit in the combiner, though the UDF itself may only
be executed in the reducer. "

> Combiner optimizations extended to nested foreach statements as well
> --------------------------------------------------------------------
>                 Key: PIG-419
>                 URL: https://issues.apache.org/jira/browse/PIG-419
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Anand Murugappan
>            Assignee: Thejas M Nair
> While Pig 2.0 seems to have optimized foreach statements by using the combiner more aggressively,
nested foreach statements lack this functionality. Given that several of our projects use
nested foreach statements, we would like to see the optimizations extended to those cases
as well. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message