hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-287) Speed up SequenceFile sort with memory reduction
Date Wed, 07 Jun 2006 22:12:33 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-287?page=comments#action_12415223 ] 

Doug Cutting commented on HADOOP-287:

This looks good, but it doesn't pass unit tests any longer.  In particular, the following
fails for me:

ant -Dtestcase=TestSequenceFile test

This results in something like the following for me:

java.lang.RuntimeException: wrong key at 2838
	at org.apache.hadoop.io.TestSequenceFile.checkSort(TestSequenceFile.java:146)
	at org.apache.hadoop.io.TestSequenceFile.testSequenceFile(TestSequenceFile.java:53)

> Speed up SequenceFile sort with memory reduction
> ------------------------------------------------
>          Key: HADOOP-287
>          URL: http://issues.apache.org/jira/browse/HADOOP-287
>      Project: Hadoop
>         Type: Improvement

>   Components: io
>     Versions: 0.3.2
>     Reporter: Benjamin Reed
>  Attachments: zoom-sort.patch
> I replaced the merge sort with a quick sort and it yielded approx 30% improvement in
sort time. It also reduced the memory requirement for sorting because the sort is done in

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message