pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ken Goodhope (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2153) POProject throws an error with tuples containing a single non-tuple field
Date Wed, 06 Jul 2011 21:49:18 GMT

    [ https://issues.apache.org/jira/browse/PIG-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060860#comment-13060860

Ken Goodhope commented on PIG-2153:

That makes sense, and if it is still the case it would mean the fix needs to occur in the
LoadFunc and not POProject.  This is also consistent with the original comments by Daniel
Dae for PIG-1890. AvroStorage has always included the wrapping tuple as part of the schema.
In most cases the outer tuple isn't really a wrapper, but a record with multiple fields and
that works fine.  Later tonight I will take a look and see what changes I need to make at
the LoadFunc level.  I am still perplexed why the incorrect behavior used to work.  Thanks
again Pradeep.

> POProject throws an error with tuples containing a single non-tuple field
> -------------------------------------------------------------------------
>                 Key: PIG-2153
>                 URL: https://issues.apache.org/jira/browse/PIG-2153
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1
>            Reporter: Ken Goodhope
> When POProject.getNext(tuple) processes a tuple with one field, the field is pulled out.
 If that field is not a tuple, a cast exception is thrown.  This is happening in the folliwing
block of code at line 401.
>            if(columns.size() == 1) {
>                 try{
>                     ret = inpValue.get(columns.get(0));
>                 ...
>            res.result = (Tuple)ret;
> I am seeing this error in a unit test that is loading an array of floats.  The LoadFunc
is converting the array to bag, and wrapping the bag in a tuple.  
> ({(3.3),(1.2),(5.6)})
> This results on POProject attempting to cast the bag to a tuple.  Looking at the code,
it appears that if I wrapped the previous tuple in another tuple, then it would work.
> (({(3.3),(1.2),(5.6)}))
> In this case it would work because POProject would extract the first inner tuple and
return it.  But this would require the LoadFunc to check for tuples with a single non-tuple
field and only wrap those.
> This could be fixed by first checking that the tuple does actually wrap another tuple.
>            if(columns.size() == 1 && inpValue.getType(0) == DataType.TUPLE) {...
> I don't know the original intent of this code well enough to say this is the appropriate
fix or not.  Hoping someone with more Pig experience can help here.  Right now this is preventing
the unit tests in AvroStorage from working.  I can change the unit test, but I think in this
case the unit test is catching a real bug.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message