Mailing-List: contact pig-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: pig-dev@incubator.apache.org
Message-ID: <472170202.1207667485688.JavaMail.jira@brutus>
Date: Tue, 8 Apr 2008 08:11:25 -0700 (PDT)
From: "Shravan Matthur Narayanamurthy (JIRA)" <jira@apache.org>
To: pig-dev@incubator.apache.org
Subject: [jira] Commented: (PIG-162) Rework mapreduce submission and
 monitoring
In-Reply-To: <1695233570.1205957904509.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/PIG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586832#action_12586832 ] 

Shravan Matthur Narayanamurthy commented on PIG-162:
----------------------------------------------------

Well anything that is non-blocking and occurs right after the cogroup will be pushed into the reduce phase. For ex.,

A = load 'A' as (x,y);
B = group A by $0;
C = foreach B generate group,COUNT(A);
D = filter C by $1>10;
store D into 'D';

Here the foreach and filter will be pushed into the reduce phase so that there is only one Map-Reduce job. The load and local-rearrange will occurr inside the map and the package, foreach & filter will be in the reduce phase.

Hope this helps and gets me more help :)

> Rework mapreduce submission and monitoring
> ------------------------------------------
>
>                 Key: PIG-162
>                 URL: https://issues.apache.org/jira/browse/PIG-162
>             Project: Pig
>          Issue Type: Sub-task
>         Environment: This bug tracks works to rework the submission and monitoring interface to map reduce as described in  http://wiki.apache.org/pig/PigTypesFunctionalSpec
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: split.png
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.