pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1720) java.lang.NegativeArraySizeException during Quicksort
Date Wed, 17 Nov 2010 17:16:14 GMT

    [ https://issues.apache.org/jira/browse/PIG-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933032#action_12933032
] 

Thejas M Nair commented on PIG-1720:
------------------------------------

Unit tests have completed successfully.


> java.lang.NegativeArraySizeException during Quicksort
> -----------------------------------------------------
>
>                 Key: PIG-1720
>                 URL: https://issues.apache.org/jira/browse/PIG-1720
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.8.0
>            Reporter: Viraj Bhat
>            Assignee: Thejas M Nair
>             Fix For: 0.8.0
>
>         Attachments: PIG-1720.1.patch
>
>
> I have a simple script, which I attempt to run with Pig 0.8
> The script looks like this:
> {code}
> clicks = load 'pairs.out' using PigStorage('\u0001') as (user,query,id);
> query0 = group clicks by query;
> query1 = foreach query0 {
>         distinct_id = DISTINCT clicks.id; 
>         distinct_user = DISTINCT clicks.user;
>         count = COUNT(distinct_user);
>         generate count as count,group,distinct_id; }
> store query1 into 'query.out' using PigStorage('|');
> {code}
> The Mapper's fail in the quick sort phase with the following error message:
> {quote}
> java.lang.NegativeArraySizeException at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinInterSedesDatum(BinInterSedes.java:782)
at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinSedesTuple(BinInterSedes.java:662)
at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:623)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:78)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:995) at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:95)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1281)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1182) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:608)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:676) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
at org.apache.hadoop.mapred.Child$4.run(Child.java:242) at java.security.AccessController.doPrivileged(Native
Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1045)
at org.apache.hadoop.mapred.Child.main(Child.java:236) 
> {quote}
> Viraj

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message