pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anup Ahire (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4023) BigDec/Int sort is broken
Date Wed, 25 Jun 2014 19:17:24 GMT

    [ https://issues.apache.org/jira/browse/PIG-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043940#comment-14043940
] 

Anup Ahire commented on PIG-4023:
---------------------------------

Thanks Daniel !

> BigDec/Int sort is broken
> -------------------------
>
>                 Key: PIG-4023
>                 URL: https://issues.apache.org/jira/browse/PIG-4023
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.12.0
>            Reporter: Anup Ahire
>            Assignee: Anup Ahire
>             Fix For: 0.14.0
>
>         Attachments: PIG-4023.patch
>
>
> BigDec/Int sort seems broken. I am getting following errors.
> java.lang.ClassCastException: java.math.BigInteger incompatible with java.lang.Double
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigBigIntegerRawComparator.compare(PigBigIntegerRawComparator.java:99)
> 	at java.util.Arrays.binarySearch(Arrays.java:700)
> 	at java.util.Arrays.binarySearch(Arrays.java:305)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:81)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:60)
> 	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
> 	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> 	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:122)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
> 	----------------------------------------------------------------------------
> 	java.lang.ClassCastException: java.math.BigDecimal incompatible with java.lang.Double
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigBigDecimalRawComparator.compare(PigBigDecimalRawComparator.java:99)
> 	at java.util.Arrays.binarySearch(Arrays.java:700)
> 	at java.util.Arrays.binarySearch(Arrays.java:305)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:81)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:60)
> 	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
> 	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> 	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:122)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
> 	
> 	PigBig(Decimal/Intger)RawComparator wrongly casts BigInteger and BigDecimal to Double.
> Also, Big(Decimal/Integer)Writable classes are not using buffer offset and length while
initializing ByteArrayInputStream in compare method



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message