hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gianmarco De Francisci Morales (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-566) Dump and store outputs do not match for PigStorage
Date Wed, 12 May 2010 15:28:41 GMT

     [ https://issues.apache.org/jira/browse/PIG-566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gianmarco De Francisci Morales updated PIG-566:
-----------------------------------------------

    Attachment: PIG-566.patch

Ok, I understand.
Anyway this works only for integers, because their MAX/MIN_VALUE can be represented exactly
as a double. The same cannot be done for longs.
You can try yourself:
Integer.MIN_VALUE:  -2147483648
As a double:               -2.147483648E9
Integer.MAX_VALUE:  2147483647
As a double:                2.147483647E9

Long.MIN_VALUE:     -9223372036854775808
As a double:               -9.223372036854776E18
Long.MAX_VALUE:     9223372036854775807
As a double:                9.223372036854776E18

The double values for longs are rounded up. Adding or subtracting 1 to MAX/MIN does not make
sense for longs.
For example Long.valueOf(Long.MAX_VALUE).doubleValue() + 1 gets casted without errors.
Have a look at the test I added, TestConversions.testOverflow.
To make the test pass I had to add/subtract 10000 to the MAX/MIN in order to have an invalid
long.

So, I restored the +/-1 in Utf8StorageConverter for Integers. I added a >= / <= to avoid
converting Integer.MAX_VALUE + 1 and Integer.MIN_VALUE -1.
For long, according to me, it does not make sense, because +/-1 is not enough to make a different
double.

> Dump and store outputs do not match for PigStorage
> --------------------------------------------------
>
>                 Key: PIG-566
>                 URL: https://issues.apache.org/jira/browse/PIG-566
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.7.0, 0.8.0
>            Reporter: Santhosh Srinivasan
>            Assignee: Gianmarco De Francisci Morales
>            Priority: Minor
>             Fix For: 0.7.0, 0.8.0
>
>         Attachments: PIG-566.patch, PIG-566.patch, PIG-566.patch, PIG-566.patch
>
>
> The dump and store formats for PigStorage do not match for longs and floats.
> {code}
> grunt> y = foreach x generate {(2985671202194220139L)};
> grunt> describe y;
> y: {{(long)}}
> grunt> dump y;
> ({(2985671202194220139L)})
> grunt> store y into 'y';
> grunt> cat y
> {(2985671202194220139)}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message