pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Reed (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-62) Need to add pig script and input dirs (in clear text format) to jobconf
Date Fri, 25 Jan 2008 15:37:36 GMT

    [ https://issues.apache.org/jira/browse/PIG-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562541#action_12562541
] 

Benjamin Reed commented on PIG-62:
----------------------------------

I'm not a fan of this issue, but the patch is simple, so I'm all for committing it if Runping
is satisfied.

The weird thing is that the query key will contain:

a = load 'foo';
b = group a by $1;
c = foreach b generate group, sum($1);
d = foreach a generate $1,myfunc($2);
store d into 'blah';

when the map/reduce job is run for the store d, even though d was really only derived from

a = load 'foo';
d = foreach a generate $1,myfunc($2);
store d into 'blah';

But, to get the real value is a bit difficult and is of limited use since there isn't a 1-1
correspondence between logical plan and map/reduce job.

> Need to add pig script and input dirs (in clear text format) to jobconf
> -----------------------------------------------------------------------
>
>                 Key: PIG-62
>                 URL: https://issues.apache.org/jira/browse/PIG-62
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Runping Qi
>         Attachments: PIG-62-v01.patch
>
>
> We'd like to know what pig query each pig related map/reduce job is for, and what input
dir the pig job use.
> It will be nice to add pig script and input dirs (in clear text format) to the jobconfs
of the jobs created for a pig script.
> .

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message