hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Douglas <chri...@yahoo-inc.com>
Subject Re: Stackoverflow
Date Wed, 04 Jun 2008 18:39:44 GMT
The pivot selection is the median of the first, middle, and last  
elements; it should be the best choice for sorted data. It's still  
possible to pick bad pivots, but data that forces hundreds of  
consecutive bad pivot selections should be exceedingly rare. -C

On Jun 4, 2008, at 9:24 AM, Doug Cutting wrote:

> Andreas Kostyrka wrote:
>> java.lang.StackOverflowError
>> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare 
>> (MapTask.java:494)
>> 	at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:29)
>> 	at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:58)
>> 	at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:82)
>
> Quicksort is known to cause stack overflows when sorting large,  
> already-sorted data.  Could that be the issue here?
>
> http://safari.oreilly.com/0201361205/ch07lev1sec3
> http://www.finesse.demon.co.uk/steven/sorting.html#quicksort
> http://www.seeingwithc.org/topic2html.html
>
> Doug
>


Mime
View raw message