From hadoop-dev-return-4271-apmail-lucene-hadoop-dev-archive=lucene.apache.org@lucene.apache.org Sat Oct 14 02:48:51 2006 Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 83022 invoked from network); 14 Oct 2006 02:48:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 14 Oct 2006 02:48:51 -0000 Received: (qmail 75797 invoked by uid 500); 14 Oct 2006 02:48:47 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 75751 invoked by uid 500); 14 Oct 2006 02:48:47 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 75735 invoked by uid 99); 14 Oct 2006 02:48:47 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Oct 2006 19:48:47 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [209.237.227.198] (HELO brutus.apache.org) (209.237.227.198) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Oct 2006 19:48:46 -0700 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 569F6714314 for ; Fri, 13 Oct 2006 19:48:01 -0700 (PDT) Message-ID: <15964233.1160794081352.JavaMail.jira@brutus> Date: Fri, 13 Oct 2006 19:48:01 -0700 (PDT) From: "Benjamin Reed (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Commented: (HADOOP-287) Speed up SequenceFile sort with memory reduction In-Reply-To: <30420683.1149716489884.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/HADOOP-287?page=comments#action_12442138 ] Benjamin Reed commented on HADOOP-287: -------------------------------------- [[ Old comment, sent by email on Thu, 8 Jun 2006 07:32:09 -0700 ]] Duh! Sorry Doug. Stupid error. I didn't realize you had unit tests. I've fixed it. I'll run the tests now. ben > Speed up SequenceFile sort with memory reduction > ------------------------------------------------ > > Key: HADOOP-287 > URL: http://issues.apache.org/jira/browse/HADOOP-287 > Project: Hadoop > Issue Type: Improvement > Components: io > Affects Versions: 0.3.2 > Reporter: Benjamin Reed > Assigned To: Doug Cutting > Attachments: s.patch, zoom-sort.patch, zoom-sort.patch > > > I replaced the merge sort with a quick sort and it yielded approx 30% improvement in sort time. It also reduced the memory requirement for sorting because the sort is done in place. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira