pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2659) add source location of the aliases in the physical plan
Date Tue, 08 May 2012 18:35:49 GMT

    [ https://issues.apache.org/jira/browse/PIG-2659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270689#comment-13270689

Daniel Dai commented on PIG-2659:

That's awesome, with this, we can do:

1. Include line number for most backend exceptions
2. Include line number in explain
3. Include line number in stats
4. Use line number to generate job name
5. Make use of location in visualizer (PIG-2659)

Can you elaborate what "M: A[1,4],A[3,4],B[2,4] C: A[3,4],B[2,4] R: A[3,4]" means?

If we merge operators, we shall annotate how this operator come from, this can be done later.

Also be sure to remove white space changes when you checkin (svn diff -x --ignore-all-space).

Can you also add some tests?
> add source location of the aliases in the physical plan
> -------------------------------------------------------
>                 Key: PIG-2659
>                 URL: https://issues.apache.org/jira/browse/PIG-2659
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>            Reporter: Julien Le Dem
>            Assignee: Julien Le Dem
>         Attachments: PIG-2659.patch
> The goal is to provide better information about what is actually running in a job.
> In particular when alias names are being reused.
> For example with the following script:
> {code}
> A = LOAD 'foo' using PigStorage();
> B = GROUP A BY $0;
> STORE A INTO 'bar';
> {code}
> The job conf will contain the following information
> {code}
> pig.alias.location: M: A[1,4],A[3,4],B[2,4] C: A[3,4],B[2,4] R: A[3,4]
> {code}
> A caveat is that the Logical Plan Optimizer throws away the original information when
merging Logical Operators.
> this is already the case today with pig.alias 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message