hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (PIG-581) Pig should enable an option to disable the use of combiner optimizer
Date Wed, 31 Dec 2008 22:43:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Pradeep Kamath resolved PIG-581.

    Resolution: Invalid

There is already a property "pig.exec.nocombiner" which if true, disables the use of the combiner
optimizer. So the user can specify this as follows:
java -cp <path to pig.jar>:<dir containing hadoopsite.xml> -Dpig.exec.nocombiner=true
org.apache.pig.Main <pigscript>

> Pig should enable an option to disable the use of combiner optimizer
> --------------------------------------------------------------------
>                 Key: PIG-581
>                 URL: https://issues.apache.org/jira/browse/PIG-581
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>             Fix For: types_branch
> There are some cases where a combiner optimization chosen by Pig may actually be slower
than the non optimized version. For example, the use of combiner to address the issue reported
in https://issues.apache.org/jira/browse/PIG-580 could result in slower execution IF the distinct
on groups of values does not actually shrink those groups. This is however very data dependent
and the user may know before hand that this might be the case and may wish to disable the
use of the optimizer. Pig should enable an option to do so.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message