pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Carey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3323) AVRO: default value not stored in file when given as paramter to AvroStorage
Date Tue, 21 May 2013 04:25:16 GMT

    [ https://issues.apache.org/jira/browse/PIG-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662666#comment-13662666
] 

Scott Carey commented on PIG-3323:
----------------------------------

Viraj, If you think the documentation for Avro needs enhancement or clarification, please
file a bug there explaining how it could be enhanced.  Thanks!

I'll have a look at PIG-3331.
                
> AVRO: default value not stored in file when given as paramter to AvroStorage
> ----------------------------------------------------------------------------
>
>                 Key: PIG-3323
>                 URL: https://issues.apache.org/jira/browse/PIG-3323
>             Project: Pig
>          Issue Type: Bug
>          Components: piggybank
>    Affects Versions: 0.11.2
>            Reporter: Egil Sorensen
>            Assignee: Viraj Bhat
>              Labels: patch
>             Fix For: 0.12, 0.11.2
>
>
> A pig script like the below succeeds, but inspecting the resulting file I find that the
schema is stripped of the default value specification.
> {code}
> a = load ':INPATH:/types/numbers.txt' using PigStorage(':') as (intnum1000: int,id: int,intnum5:
int,intnum100: int,intnum: int,longnum: long,floatnum: float,doublenum: double);
> b2 = foreach a generate id, intnum5, intnum100;
> c2 = filter b2 by 110 <= id and id < 120;
> describe c2;
> dump c2;
> store c2 into ':OUTPATH:.intermediate_2' USING org.apache.pig.piggybank.storage.avro.AvroStorage('
> {
>    "debug" : 5,
>    "schema" : {  
>       "name" : "schema_2",
>       "type" : "record",
>       "fields" : [
>          {  
>             "name" : "id",
>             "type" : [
>                "null",
>                "int"
>             ]
>          },
>          {  
>             "name" : "intnum5",
>             "type" : [
>                "null",
>                "int"
>             ]
>          },
>          {
>             "name" : "intnum100",
>             "type" : [
>                "null",
>                "int"
>             ],
>             "default" : 0
>          }
>       ]
>    }
> }
> ');
> {code}
> BTW, the documentation on https://cwiki.apache.org/PIG/avrostorage.html is mute on the
subject of defaults, so first question is: is my expectation that the default is to be written
to file not correct?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message