hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1610) 'union onschema' does handle some cases involving 'namespaced' column names in schema
Date Wed, 15 Sep 2010 19:52:32 GMT

    [ https://issues.apache.org/jira/browse/PIG-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909859#action_12909859
] 

Thejas M Nair commented on PIG-1610:
------------------------------------

Richard pointed out an issue with the patch where the schema of 'union onschema' differs with
different order of relation in the statement. The case is like -
{code}
l = load 'x' as (c, nm::c);
f = load 'y' as (i,j);

u = union onschema f,l;
describe u;
u: {i: bytearray,j: bytearray,c: bytearray}

u = union onschema l,f;
describe u;
u: {c: bytearray,nm::c: bytearray,i: bytearray,j: bytearray}
{code}

Another issue found with the feature is that the schema of union is null when a column in
one of the relations has a complex type with null inner schema.

I will submit another patch with fix for these issues.



> 'union onschema' does handle some cases involving 'namespaced' column names in schema
> -------------------------------------------------------------------------------------
>
>                 Key: PIG-1610
>                 URL: https://issues.apache.org/jira/browse/PIG-1610
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.8.0
>
>         Attachments: PIG-1610.1.patch
>
>
> case 1:
> grunt> describe f;              
> f: {l1::a: bytearray,l1::b: bytearray}
> grunt> describe l1;
> l1: {a: bytearray,b: bytearray}
> grunt> dump f;
> (1,11)
> (2,22)
> (3,33)
> grunt> dump l1;
> (1,11)
> (2,22)
> (3,33)
> grunt> u = union onschema f, l1;
> grunt> describe u;
> u: {l1::a: bytearray,l1::b: bytearray}
> -- the dump u gives incorrect results
> grunt> dump u;     
> (,)
> (,)
> (,)
> (1,11)
> (2,22)
> (3,33)
> case 2:
> grunt> u = union onschema l1, f;
> grunt> describe u;
> 2010-09-13 15:11:13,877 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1108: Duplicate
schema alias: l1::a
> Details at logfile: /Users/tejas/pig_unions_err2/trunk/pig_1284410413970.log

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message