hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1368) Utf8StorageConvertor's bytesToTuple and bytesToBag methods need to be tightened for corner cases
Date Thu, 08 Apr 2010 18:47:37 GMT
Utf8StorageConvertor's bytesToTuple and bytesToBag methods need to be tightened for corner
cases
------------------------------------------------------------------------------------------------

                 Key: PIG-1368
                 URL: https://issues.apache.org/jira/browse/PIG-1368
             Project: Pig
          Issue Type: Improvement
    Affects Versions: 0.7.0
            Reporter: Pradeep Kamath


Consider the following data:
1\t ( hello , bye ) \n
1\t( hello , bye )a\n
2 \t (good , bye)\n

The following script gives the results below:
a = load 'junk' as (i:int, t:tuple(s:chararray, r:chararray)); dump a;

(1,( hello , bye ))
(1,( hello , bye ))
(2,(good , bye))

The current bytesToTuple implementation discards leading and trailing characters before the
tuple delimiters and parses the tuple out - I think instead it should treat any leading and
trailing characters (including space) near the delimiters as an indication of a malformed
tuple and return null.

Also in the code, consumeBag() should handle the special case of {} and not delegate the handling
to consumeTuple(). 

In consumeBag() null tuples should not be skipped.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message