pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-2970) Nested foreach getting incorrect schema when having unrelated inner query
Date Fri, 05 Apr 2013 22:17:15 GMT

     [ https://issues.apache.org/jira/browse/PIG-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai updated PIG-2970:
----------------------------

    Attachment: PIG-2970-0.patch

I feel the fix is too complex. "explain" can figure out the right schema but "describe" cannot.
The difference is "explain" does the optimization but "describe" does not. We only need to
bring the optimization to "describe" to fix the issue. I attach a temporary fix, but we only
need "optimization" part of "compilePp". So a better fix is separate "compilePp" and only
do the optimization part before "describe". I will attach a patch shortly.
                
> Nested foreach getting incorrect schema when having unrelated inner query
> -------------------------------------------------------------------------
>
>                 Key: PIG-2970
>                 URL: https://issues.apache.org/jira/browse/PIG-2970
>             Project: Pig
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.10.0
>            Reporter: Koji Noguchi
>            Assignee: Koji Noguchi
>            Priority: Minor
>             Fix For: 0.12
>
>         Attachments: PIG-2970-0.patch, pig-2970-trunk-v01.txt, pig-2970-trunk-v02.txt
>
>
> While looking at PIG-2968, hit a weird error message.
> {noformat}
> $ cat -n test/foreach2.pig
>      1  daily = load 'nyse' as (exchange, symbol);
>      2  grpd = group daily by exchange;
>      3  unique = foreach grpd {
>      4          sym = daily.symbol;
>      5          uniq_sym = distinct sym;
>      6          --ignoring uniq_sym result
>      7          generate group, daily;
>      8  };
>      9  describe unique;
>     10  zzz = foreach unique generate group;
>     11  explain zzz;
> % pig -x local -t ColumnMapKeyPrune test/foreach2.pig
> ...
> unique: {symbol: bytearray}
> 2012-10-12 16:55:44,226 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1025: 
> <file test/foreach2.pig, line 10, column 30> Invalid field projection. Projected
field [group] does not exist in schema: symbol:bytearray.
> ...
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message