mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karan Jindal" <karan_jin...@students.iiit.ac.in>
Subject Re: Re : Reg: Maximum Split size in Random Forest
Date Thu, 10 Jun 2010 04:54:22 GMT
Hi jake,

I am assuming that by hitting u mean calling that function
Reporter.progress(). But in which part the code this function needs to be
called?
deneche abdelhakim can u try {Since, I don't know anything about how did u
code RF.} what jake suggested and let me know whether that work or not?

--Karan


> On Tue, Jun 8, 2010 at 9:19 PM, deneche abdelhakim
> <a_deneche@yahoo.fr>wrote:
>
>> mapred.max.split.size controls how many partitions will be generated
>> from
>> the data.
>> the current implementation of random forest is pretty memory intensive,
>> and
>> because all the work is done in the mappers' close method, when the data
>> is
>> Big, Hadoop just thinks that the mappers have failed (I will solve this
>> problem some day).
>>
>
> By periodically hitting Reporter.progress() in the long-lived mapper, this
> typically fixes this.
>
>   -jake
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


Mime
View raw message