pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Viraj Bhat (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-3318) AVRO: 'default value' not honored when merging schemas on load with AvroStorage
Date Wed, 12 Jun 2013 02:08:20 GMT

     [ https://issues.apache.org/jira/browse/PIG-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Viraj Bhat updated PIG-3318:
----------------------------

    Attachment: PIG-3318_3.patch
    
> AVRO: 'default value' not honored when merging schemas on load with AvroStorage
> -------------------------------------------------------------------------------
>
>                 Key: PIG-3318
>                 URL: https://issues.apache.org/jira/browse/PIG-3318
>             Project: Pig
>          Issue Type: Bug
>          Components: piggybank
>    Affects Versions: 0.11.2
>            Reporter: Viraj Bhat
>            Assignee: Viraj Bhat
>              Labels: patch
>             Fix For: 0.12, 0.11.2
>
>         Attachments: Employee3.ser, Employee4.ser, Employee6.ser, expected_testMultipleSchemasWithDefaultValue.avro,
PIG-3318_3.patch
>
>
> Piggybank - AvroStorage. When merging multiple schemas where default values have been
specified in the avro schema; 
> The AvroStorage puts nulls in the merged data set. 
> ==> Employee3.avro <==
> {
> "type" : "record",
> "name" : "employee",
> "fields":[
>         {"name" : "name", "type" : "string", "default" : "NU"},
>         {"name" : "age", "type" : "int", "default" : 0 },
>         {"name" : "dept", "type": "string", "default" : "DU"} ] }
> ==> Employee4.avro <==
> {
> "type" : "record",
> "name" : "employee",
> "fields":[
>         {"name" : "name", "type" : "string", "default" : "NU"},
>         {"name" : "age", "type" : "int", "default" : 0},
>         {"name" : "dept", "type": "string", "default" : "DU"},
>         {"name" : "office", "type": "string", "default" : "OU"} ] }
> ==> Employee6.avro <==
> {
> "type" : "record",
> "name" : "employee",
> "fields":[
>         {"name" : "name", "type" : "string", "default" : "NU"},
>         {"name" : "lastname", "type": "string", "default" : "LNU"},
>         {"name" : "age", "type" : "int","default" : 0},
>         {"name" : "salary", "type": "int", "default" : 0},
>         {"name" : "dept", "type": "string","default" : "DU"},
>         {"name" : "office", "type": "string","default" : "OU"} ] }
> The pig script:
> employee = load 'employee{3,4,6}.ser' using org.apache.pig.piggybank.storage.avro.AvroStorage('multiple_schemas');
> describe employee;
> dump employee;
> Output Schema:
> employee: {name: chararray,age: int,dept: chararray,lastname: chararray,salary: int,office:
chararray}
> (Milo,30,DH,,,)
> (Asmya,34,PQ,,,)
> (Baljit,23,RS,,,)
> (Pune,60,Astrophysics,Warriors,5466,UTA)
> (Rajsathan,20,Biochemistry,Royals,1378,Stanford)
> (Chennai,50,Microbiology,Superkings,7338,Hopkins)
> (Mumbai,20,Applied Math,Indians,4468,UAH)
> (Praj,54,RMX,,,Champaign)
> (Buba,767,HD,,,Sunnyvale)
> (Manku,375,MS,,,New York)
> Regards
> Viraj

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message