pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-162) Rework mapreduce submission and monitoring
Date Tue, 08 Apr 2008 15:11:25 GMT

    [ https://issues.apache.org/jira/browse/PIG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586832#action_12586832

Shravan Matthur Narayanamurthy commented on PIG-162:

Well anything that is non-blocking and occurs right after the cogroup will be pushed into
the reduce phase. For ex.,

A = load 'A' as (x,y);
B = group A by $0;
C = foreach B generate group,COUNT(A);
D = filter C by $1>10;
store D into 'D';

Here the foreach and filter will be pushed into the reduce phase so that there is only one
Map-Reduce job. The load and local-rearrange will occurr inside the map and the package, foreach
& filter will be in the reduce phase.

Hope this helps and gets me more help :)

> Rework mapreduce submission and monitoring
> ------------------------------------------
>                 Key: PIG-162
>                 URL: https://issues.apache.org/jira/browse/PIG-162
>             Project: Pig
>          Issue Type: Sub-task
>         Environment: This bug tracks works to rework the submission and monitoring interface
to map reduce as described in  http://wiki.apache.org/pig/PigTypesFunctionalSpec
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: split.png

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message