Return-Path: Delivered-To: apmail-lucene-hadoop-commits-archive@locus.apache.org Received: (qmail 61142 invoked from network); 15 May 2006 19:22:03 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 15 May 2006 19:22:03 -0000 Received: (qmail 51947 invoked by uid 500); 15 May 2006 19:22:03 -0000 Delivered-To: apmail-lucene-hadoop-commits-archive@lucene.apache.org Received: (qmail 51897 invoked by uid 500); 15 May 2006 19:22:03 -0000 Mailing-List: contact hadoop-commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-commits@lucene.apache.org Received: (qmail 51888 invoked by uid 99); 15 May 2006 19:22:03 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 May 2006 12:22:03 -0700 X-ASF-Spam-Status: No, hits=-9.4 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [209.237.227.194] (HELO minotaur.apache.org) (209.237.227.194) by apache.org (qpsmtpd/0.29) with SMTP; Mon, 15 May 2006 12:22:02 -0700 Received: (qmail 60957 invoked by uid 65534); 15 May 2006 19:21:42 -0000 Message-ID: <20060515192142.60941.qmail@minotaur.apache.org> Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r406722 - /lucene/hadoop/trunk/src/examples/org/apache/hadoop/examples/Sort.java Date: Mon, 15 May 2006 19:21:41 -0000 To: hadoop-commits@lucene.apache.org From: cutting@apache.org X-Mailer: svnmailer-1.0.8 X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Author: cutting Date: Mon May 15 12:21:40 2006 New Revision: 406722 URL: http://svn.apache.org/viewcvs?rev=406722&view=rev Log: Set number of reduce tasks to be the number of reduce nodes. This results in optimal sort performance. Contributed by Owen. Modified: lucene/hadoop/trunk/src/examples/org/apache/hadoop/examples/Sort.java Modified: lucene/hadoop/trunk/src/examples/org/apache/hadoop/examples/Sort.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/examples/org/apache/hadoop/examples/Sort.java?rev=406722&r1=406721&r2=406722&view=diff ============================================================================== --- lucene/hadoop/trunk/src/examples/org/apache/hadoop/examples/Sort.java (original) +++ lucene/hadoop/trunk/src/examples/org/apache/hadoop/examples/Sort.java Mon May 15 12:21:40 2006 @@ -69,7 +69,7 @@ int num_maps = cluster.getTaskTrackers() * jobConf.getInt("test.sort.maps_per_host", 10); int num_reduces = cluster.getTaskTrackers() * - jobConf.getInt("test.sort.reduces_per_host", 10); + jobConf.getInt("test.sort.reduces_per_host", cluster.getMaxTasks()); List otherArgs = new ArrayList(); for(int i=0; i < args.length; ++i) { try { @@ -109,7 +109,7 @@ cluster.getTaskTrackers() + " nodes to sort from " + jobConf.getInputPaths()[0] + " into " + - jobConf.getOutputPath() + "."); + jobConf.getOutputPath() + " with " + num_reduces + " reduces."); Date startTime = new Date(); System.out.println("Job started: " + startTime); JobClient.runJob(jobConf);