hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-1617) 'group all' should always use one reducer
Date Mon, 20 Sep 2010 11:52:33 GMT

     [ https://issues.apache.org/jira/browse/PIG-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Thejas M Nair updated PIG-1617:

    Attachment: PIG-1617.1.patch

PIG-1617.1.patch- Patch sets parallelism of LOCogroup to 1 for group on a constant (including
'group all')
Passes unit tests and test patch. Ready for review.

> 'group all' should always use one reducer
> -----------------------------------------
>                 Key: PIG-1617
>                 URL: https://issues.apache.org/jira/browse/PIG-1617
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.8.0
>         Attachments: PIG-1617.1.patch
> 'group all' sends all rows to a single reducer, it does not make sense to spawn more
than one reducer for it. But if higher value of parallelism is specified or if the input is
large enough so that changes in PIG-1249 result in larger value being set, there are additional
reducers spawned that don't do anything useful.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message