hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-26) distinct does not work on Bags that have spilled to disk.
Date Fri, 09 Nov 2007 19:27:50 GMT

    [ https://issues.apache.org/jira/browse/PIG-26?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541413
] 

Doug Cutting commented on PIG-26:
---------------------------------

Perhaps I should have been more verbose.  The HowToContribute page in the wiki advises against
making formatting changes unless the patch is exclusively dedicated to formatting.  The point
is to make patches maximally easy to review, so that reviewers can easily focus on the changes
of function.

If there are lots of tabs in the codebase, then someone should file an issue for that and
then fix it in one independent sweep.

> distinct does not work on Bags that have spilled to disk.
> ---------------------------------------------------------
>
>                 Key: PIG-26
>                 URL: https://issues.apache.org/jira/browse/PIG-26
>             Project: Pig
>          Issue Type: Bug
>          Components: data
>    Affects Versions: 0.0.0, 0.1.0, site
>            Reporter: Benjamin Reed
>            Assignee: Benjamin Reed
>         Attachments: distinct-test.patch, distinct.patch
>
>
> If you call distinct on a bag that has spilled to disk, you get the following error:
> java.lang.NullPointerException
>         at org.apache.pig.data.BigDataBag$FileMerger$1.compare(BigDataBag.java:288)
>         at org.apache.pig.data.BigDataBag$FileMerger$1.compare(BigDataBag.java:280)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:594)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:572)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:274)
>         at java.util.PriorityQueue.add(PriorityQueue.java:251)
>         at org.apache.pig.data.BigDataBag$FileMerger.<init>(BigDataBag.java:304)
>         at org.apache.pig.data.BigDataBag.doSorting(BigDataBag.java:167)
>         at org.apache.pig.data.BigDataBag.content(BigDataBag.java:211)
>         at org.apache.pig.test.TestDataModel.testBigDataBag(TestDataModel.java:343)
>         at org.apache.pig.test.TestDataModel.testBigDataBagOnDisk(TestDataModel.java:210)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message