lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera" <ser...@gmail.com>
Subject Re: Explanation on RAMBufferSizeMB
Date Sat, 22 Mar 2008 04:24:24 GMT
Thanks a lot for your responses on this.
If I'll have more results on this issue, I'll post them back here.

Shai

On Fri, Mar 21, 2008 at 11:38 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

>
> Shai Erera wrote:
> > What do you mean by "does your test do any merging"?
> > All I do is create IndexWriter w/ the RAM and MBD settings as I've
> > described
> > before. Then I just call addDocument. At the end I call optimize()
> > (it is a
> > one time created index, after that I need it optimized for search).
> > I guess Lucene performs segment merging in between, although I think I
> > noticed that before optimize is called, there were 10 segments (CFS
> > files)
> > of size 200MB each, which matches the default merge factor setting
> > to 10 (if
> > I'm not mistaken). Does it mean that no segments were merged during
> > the
> > indexing process?
>
> It sounds likely that Lucene didn't do any of its own segment merging
> before you called optimize.  So this is a good example: if your ram
> buffer size had been smaller, probably more than 10 segments would be
> flushed which would have kicked off a merge.  So even though
> throughput tails off as you increase RAM buffer size, larger RAM
> buffers result in fewer merges taking place, so for a large enough
> index it should be worthwhile having a larger RAM buffer size.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>


-- 
Regards,

Shai Erera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message