pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1341) BinStorage cannot convert DataByteArray to Chararray and results in FIELD_DISCARDED_TYPE_CONVERSION_FAILED
Date Wed, 21 Apr 2010 01:13:49 GMT

    [ https://issues.apache.org/jira/browse/PIG-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859157#action_12859157
] 

Ashutosh Chauhan commented on PIG-1341:
---------------------------------------

I think BinStorage is an internal way of moving data around in Pig and it should be treated
that way. I think we should discourage its usage to user. Otherwise, we need to add capabilities
as the one requested here. Important impact of making such a change is that we can't  then
swap out BinStorage with other storage mechanisms. If Avro (or protobuf or whatever) proved
to be a better replacement for BinStorage, then we cant just swap them in place of BinStorage,
unless we add to them all the capabilities that BinStorage has. Therefore, I suggest to keep
capabilities of BinStorage to minimal.  

> BinStorage cannot convert DataByteArray to Chararray and results in FIELD_DISCARDED_TYPE_CONVERSION_FAILED
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-1341
>                 URL: https://issues.apache.org/jira/browse/PIG-1341
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>            Reporter: Viraj Bhat
>            Assignee: Richard Ding
>         Attachments: PIG-1341.patch
>
>
> Script reads in BinStorage data and tries to convert a column which is in DataByteArray
to Chararray. 
> {code}
> raw = load 'sampledata' using BinStorage() as (col1,col2, col3);
> --filter out null columns
> A = filter raw by col1#'bcookie' is not null;
> B = foreach A generate col1#'bcookie'  as reqcolumn;
> describe B;
> --B: {regcolumn: bytearray}
> X = limit B 5;
> dump X;
> B = foreach A generate (chararray)col1#'bcookie'  as convertedcol;
> describe B;
> --B: {convertedcol: chararray}
> X = limit B 5;
> dump X;
> {code}
> The first dump produces:
> (36co9b55onr8s)
> (36co9b55onr8s)
> (36hilul5oo1q1)
> (36hilul5oo1q1)
> (36l4cj15ooa8a)
> The second dump produces:
> ()
> ()
> ()
> ()
> ()
> It also throws an error message: FIELD_DISCARDED_TYPE_CONVERSION_FAILED 5 time(s).
> Viraj

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message