mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Taste and -XX:+NewRatio
Date Thu, 12 Nov 2009 20:48:04 GMT
Hello,

----- Original Message ----

> From: Sean Owen <srowen@gmail.com>
> 
> Yes, the advice is intended to be "make it larger than the default".
> As I recall the default was a lot lower, like 4. This was on a Mac
> though. I am happy to make the default advice more like 12. The point
> is that almost all the memory is long-lived data structures so almost

You are referring to objects mostly references from DataModel?  Things like preferences and
preference arrays, sets with user and item iDs and such?  Or some other ones that I'm forgetting?

> all the heap should be allocated to this. It ought to be tuned since
> the ratio of long-lived to short-lived objects is unusually high. YMMV
> but I'm sure the right value is at least 9.

Does this hold true even if you refresh the whole model fairly frequently, say every 5, 10,
15, or 60 minutes?

Thanks,
Otis

> On Thu, Nov 12, 2009 at 7:40 PM, Otis Gospodnetic
> wrote:
> > Hello,
> >
> > Something on http://lucene.apache.org/mahout/taste.html#performance caught my 
> attention:
> >
> > -XX:+NewRatio=9: Increase heap allocated to 'old' objects, which is most of 
> them in this framework
> >
> > So I looked up this java parameter on 
> http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp :
> >
> >
> > -XX:NewRatio=2
> > Ratio of new/old generation sizes. [Sparc -client: 8; x86 -server: 8; x86 
> -client: 12.]-client: 4 (1.3) 8 (1.3.1+), x86: 12]
> >
> > If I'm reading this correctly, the larger the number, the bigger portion of 
> the heap is used for "old" objects.
> > Also, it looks like different platforms have different defaults, so maybe 
> advising a single "9" value is not the right thing to do?
> > For example, it looks like on x86 the default is 12, so using NewRatio=9 would 
> actually decrease the space for old objects.
> >
> > I'd love to hear from anyone who actually knows what this means, how it should 
> be used, and if anyone has actually used different values with a large Taste 
> instance.
> >
> > Thanks,
> > Otis
> > --
> > Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
> >
> >


Mime
View raw message