hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-287) Speed up SequenceFile sort with memory reduction
Date Tue, 01 Aug 2006 17:30:17 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-287?page=comments#action_12424941 ] 
Owen O'Malley commented on HADOOP-287:

This patch broke the build. In particular, the org.apache.hadoop.mapred.TestMiniMRLocalFS
no longer finishes. When the job becomes stuck, the reduces are still running. I assume they
are stuck in an infinite loop somewhere. 

> Speed up SequenceFile sort with memory reduction
> ------------------------------------------------
>                 Key: HADOOP-287
>                 URL: http://issues.apache.org/jira/browse/HADOOP-287
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 0.3.2
>            Reporter: Benjamin Reed
>         Assigned To: Doug Cutting
>             Fix For: 0.5.0
>         Attachments: s.patch, zoom-sort.patch, zoom-sort.patch
> I replaced the merge sort with a quick sort and it yielded approx 30% improvement in
sort time. It also reduced the memory requirement for sorting because the sort is done in

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message