Return-Path: Delivered-To: apmail-mahout-user-archive@www.apache.org Received: (qmail 16891 invoked from network); 10 Jun 2010 04:55:11 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Jun 2010 04:55:11 -0000 Received: (qmail 89475 invoked by uid 500); 10 Jun 2010 04:55:11 -0000 Delivered-To: apmail-mahout-user-archive@mahout.apache.org Received: (qmail 89317 invoked by uid 500); 10 Jun 2010 04:55:09 -0000 Mailing-List: contact user-help@mahout.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@mahout.apache.org Delivered-To: mailing list user@mahout.apache.org Received: (qmail 89307 invoked by uid 99); 10 Jun 2010 04:55:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jun 2010 04:55:08 +0000 X-ASF-Spam-Status: No, hits=-0.5 required=10.0 tests=AWL,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of karan_jindal@students.iiit.ac.in designates 121.242.23.201 as permitted sender) Received: from [121.242.23.201] (HELO students.iiit.ac.in) (121.242.23.201) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jun 2010 04:55:03 +0000 MailScanner-NULL-Check: 1276750463.11694@rCWzhTEQZYN1Do9IH49i/g Received: from students.iiit.ac.in (localhost.localdomain [127.0.0.1]) by students.iiit.ac.in (8.13.8/8.13.8) with ESMTP id o5A4sMND016434 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 10 Jun 2010 10:24:22 +0530 Received: (from apache@localhost) by students.iiit.ac.in (8.13.8/8.14.1/Submit) id o5A4sMIh016413; Thu, 10 Jun 2010 10:24:22 +0530 X-Authentication-Warning: students.iiit.ac.in: apache set sender to karan_jindal@localhost using -f Received: from 125.16.17.152 (proxying for unknown) (SquirrelMail authenticated user karan_jindal) by students.iiit.ac.in with HTTP; Thu, 10 Jun 2010 10:24:22 +0530 (IST) Message-ID: <24870.125.16.17.152.1276145662.squirrel@students.iiit.ac.in> In-Reply-To: References: <34437.125.16.17.152.1275996117.squirrel@students.iiit.ac.in> <316303.26331.qm@web26307.mail.ukl.yahoo.com> Date: Thu, 10 Jun 2010 10:24:22 +0530 (IST) Subject: Re: Re : Reg: Maximum Split size in Random Forest From: "Karan Jindal" To: user@mahout.apache.org User-Agent: SquirrelMail/1.4.8-4.el5 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on students.iiit.ac.in X-yoursite-MailScanner-Information: Please contact the IIIT Server Room for more information X-MailScanner-ID: o5A4sMND016434 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: karan_jindal@students.iiit.ac.in X-Old-Spam-Status: No, score=1.9 required=5.0 tests=ALL_TRUSTED,FH_DATE_PAST_20XX autolearn=no version=3.2.5, No Hi jake, I am assuming that by hitting u mean calling that function Reporter.progress(). But in which part the code this function needs to be called? deneche abdelhakim can u try {Since, I don't know anything about how did u code RF.} what jake suggested and let me know whether that work or not? --Karan > On Tue, Jun 8, 2010 at 9:19 PM, deneche abdelhakim > wrote: > >> mapred.max.split.size controls how many partitions will be generated >> from >> the data. >> the current implementation of random forest is pretty memory intensive, >> and >> because all the work is done in the mappers' close method, when the data >> is >> Big, Hadoop just thinks that the mappers have failed (I will solve this >> problem some day). >> > > By periodically hitting Reporter.progress() in the long-lived mapper, this > typically fixes this. > > -jake > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > > -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.