pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Schlaikjer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2715) Cannot specify schema field alias for map value via piglatin
Date Tue, 22 May 2012 22:31:41 GMT

    [ https://issues.apache.org/jira/browse/PIG-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281256#comment-13281256
] 

Andy Schlaikjer commented on PIG-2715:
--------------------------------------

I agree, though I also feel that it's too easy for java code to create an output (or load)
schema which specifies a field alias for map values. This in turn leads to problems when strict
matching of schemas is required and clients are unable to affect the alias via piglatin.

So, perhaps {{FieldSchema(String a, Schema s, byte t)}} could be hardened to prohibit this:

http://svn.apache.org/viewvc/pig/trunk/src/org/apache/pig/impl/logicalLayer/schema/Schema.java?view=markup#l131

                
> Cannot specify schema field alias for map value via piglatin
> ------------------------------------------------------------
>
>                 Key: PIG-2715
>                 URL: https://issues.apache.org/jira/browse/PIG-2715
>             Project: Pig
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.10.0
>            Reporter: Andy Schlaikjer
>
> Programmatically, pig components are able to define schema field aliases for map values,
but the same cannot be done via piglatin. For example:
> {code}
> %default INPUT_PATH 'map_value_alias_bug.tsv';
> A = LOAD '$INPUT_PATH' AS (
>   id: long,
>   -- events: [count: int] -- parse failure
>   events: [int]
> );
> DESCRIBE A;
> A = FOREACH A GENERATE
>   id,
>   -- (map[count: int]) events -- parse failure
>   -- ([count: int]) events -- parse failure
>   -- events: [count: int] -- parse failure
>   -- events AS [count: int] -- parse failure
>   -- events AS events: [count: int] -- parse failure
>   ([int]) events -- this is okay
>   ;
> DESCRIBE A;
> quit;
> {code}
> When using the [Elephant Bird project|https://github.com/kevinweil/elephant-bird/]'s
[ThriftToPig utility|https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/pig/util/ThriftToPig.java#L273]
along with a Thrift message spec containing a map field, I see schemas of the following form:
> {code}
> message: {id: long,events: map[events: int])}}
> {code}
> This same DESCRIBE output does not parse if used within piglatin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message