lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Nioche" <Julien.Nio...@lingway.com>
Subject Re: Memory Requirements
Date Thu, 13 May 2004 13:15:25 GMT
An important part of the memory is used when accessing information about
Terms. A lot of Term objects are temporarily created.
In average (and for an usual index) 64 Term objects are created for every
Term information searched. Depending on your queries this may not be a real
issue, but if you have very complex query this could be important.

The TermInfoWriter object has a field called indexinterval. By modifying it
more Terms will be stored in memory but temporary creation of Term objects
should be reduced. Performance may be a little bit improved as well because
disk access is limited + less garbage collecting

/** Expert: The fraction of terms in the "dictionary" which should be stored
* in RAM. Smaller values use more memory, but make searching slightly

* faster, while larger values use less memory and make searching slightly

* slower. Searching is typically not dominated by dictionary lookup, so

* tweaking this is rarely useful.*/

int indexInterval = 128;

--------------------------








----- Original Message -----
From: "Matt Quail" <matt@ctx.com.au>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Sent: Thursday, May 13, 2004 1:36 PM
Subject: Re: Memory Requirements


> > do you know of any method to reduce the memory consumption of lucene
> > when searching?
>
> It depends on the complexity of the search, I think. Also, I belive
> scoring might use more memory than the search itself (can anyone confirm
> this?). For example, I often use the HitCollector interface (and a
> BitSet) for queries where I  am not interested in the score.
>
> Apart from that, I'm not aware of any other methods for reducing the
> memory consumption.
>
> =Matt
>
> Sascha Ottolski wrote:
>
> > Am Donnerstag, 13. Mai 2004 12:56 schrieb Matt Quail:
> >
> >>>I noticed that  most users have +- 1G of RAM to run Lucene. Does
> >>>anyone have experiences running it on a 128MB or 256MB machine?
> >>
> >>I regularly test my app that uses Lucene by passing -Xmx8m to the
> >>JVM; this is on a box with 1G of ram, but the JVM never more than 8M.
> >>My app runs fine (though there is a little more garbage collection
> >>activity).
> >
> >
> > do you know of any method to reduce the memory consumption of lucene
> > when searching? I've just increased from 400 to --Xmx500m, since
> > sometimes OutOfMemoryExceptions occured (running suns java 1.4.2_04 and
> > lucene-1.4rc3 on an index with ca. 18 Mio entries, building an index
> > (including stored-only-fields) of 9 GB).
> >
> > I've seen that there are ways to limit the memory need for indexing and
> > optimizing(?) by reducing the MergeFactor, but that doesn't seem to
> > apply for searching :-(
> >
> > Thanks,
> >
> > Sascha
> >
> >
> >>=Matt
> >>
> >>
> >>
> >>
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> >>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
> >
> >
> > Mit freundlichen GrĂ¼ssen,
> >
> > Sascha Ottolski
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message