hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-544) Utf8StorageConverter.java does not always produce NULLs when data is malformed
Date Wed, 26 Nov 2008 00:09:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650810#action_12650810

Santhosh Srinivasan commented on PIG-544:

Another use case where scalars also generate errors:


grunt> a = load 'student_tab.data';
grunt> store a into 'student_tab.bin' using BinStorage();
grunt> a = load 'student_tab.bin' using BinStorage() as (name: int, age: int, gpa: float);
grunt> dump a;

2008-11-25 16:02:40,986 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher
- Error message from task (map) task_200809241441_24635_m_000000java.lang.RuntimeException
: Unexpected data type 74 found in stream.         at org.apache.pig.data.DataReaderWriter.readDatum(DataReaderWriter.java:115)
        at org.apache.pig.builtin.BinStorage.bytesToInteger(BinStorage.java:169)         at


> Utf8StorageConverter.java does not always produce NULLs when data is malformed
> ------------------------------------------------------------------------------
>                 Key: PIG-544
>                 URL: https://issues.apache.org/jira/browse/PIG-544
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
> It does so for scalar types but not for complext types and not for the fields inside
of the complext types.
> This is because it uses different code to parse scalar types by themselves and scalar
types inside of a complex type. It should really use the same (its own code to do so.)
> The code it is currently uses, is inside of TextDataParser.jjt and is also used to parse
constants so we need to be careful if we want to make changes to it.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message