Return-Path: Delivered-To: apmail-hadoop-core-commits-archive@www.apache.org Received: (qmail 67630 invoked from network); 4 Apr 2008 20:40:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 4 Apr 2008 20:40:32 -0000 Received: (qmail 24143 invoked by uid 500); 4 Apr 2008 20:40:32 -0000 Delivered-To: apmail-hadoop-core-commits-archive@hadoop.apache.org Received: (qmail 24122 invoked by uid 500); 4 Apr 2008 20:40:32 -0000 Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-commits@hadoop.apache.org Received: (qmail 24110 invoked by uid 99); 4 Apr 2008 20:40:32 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Apr 2008 13:40:32 -0700 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO eris.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Apr 2008 20:39:49 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id D39B51A9832; Fri, 4 Apr 2008 13:40:09 -0700 (PDT) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r644900 - in /hadoop/core/trunk: CHANGES.txt src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeMapRed.java Date: Fri, 04 Apr 2008 20:40:09 -0000 To: core-commits@hadoop.apache.org From: omalley@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20080404204009.D39B51A9832@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: omalley Date: Fri Apr 4 13:40:08 2008 New Revision: 644900 URL: http://svn.apache.org/viewvc?rev=644900&view=rev Log: HADOOP-3168. Reduce the amount of logging in streaming to an exponentially increasing number of records (up to 10,000 records/log). Contributed by Zheng Shao. Modified: hadoop/core/trunk/CHANGES.txt hadoop/core/trunk/src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeMapRed.java Modified: hadoop/core/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/core/trunk/CHANGES.txt?rev=644900&r1=644899&r2=644900&view=diff ============================================================================== --- hadoop/core/trunk/CHANGES.txt (original) +++ hadoop/core/trunk/CHANGES.txt Fri Apr 4 13:40:08 2008 @@ -249,6 +249,10 @@ HADOOP-3140. Doesn't add a task in the commit queue if the task hadn't generated any output. (Amar Kamat via ddas) + HADOOP-3168. Reduce the amount of logging in streaming to an + exponentially increasing number of records (up to 10,000 + records/log). (Zheng Shao via omalley) + BUG FIXES HADOOP-2195. '-mkdir' behaviour is now closer to Linux shell in case of Modified: hadoop/core/trunk/src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeMapRed.java URL: http://svn.apache.org/viewvc/hadoop/core/trunk/src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeMapRed.java?rev=644900&r1=644899&r2=644900&view=diff ============================================================================== --- hadoop/core/trunk/src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeMapRed.java (original) +++ hadoop/core/trunk/src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeMapRed.java Fri Apr 4 13:40:08 2008 @@ -511,8 +511,11 @@ String info = numRecInfo(); logprintln(info); logflush(); - //nextRecReadLog_ *= 10; - nextRecReadLog_ += 100; + if (nextRecReadLog_ < 100000) { + nextRecReadLog_ *= 10; + } else { + nextRecReadLog_ += 100000; + } } }