lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson" <erickerick...@gmail.com>
Subject Re: MergeFactor advice wanted
Date Thu, 03 May 2007 14:33:37 GMT
I don't think you're doing yourself any good by explicitly using a
RAMdirectory in the first place. If you use a simple FSDirectory, a
number of documents are added in RAM before being flushed to the
FS.

Why do you add this complexity to your code with no proof that
it does you any good? Or do you have some data that you
haven't posted? If so, please post it since I'm sure it would be
of interest to others as well as myself.

Best
Erick

On 5/3/07, Chandan Tamrakar <chandan@ccnep.com.np> wrote:
>
> What if we are using addindexes(Ram Directory) method ? it calls optimize
> function inside the function itself ?
>
> Any solution to this ?
>
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Thursday, May 03, 2007 4:03 PM
> To: java-user@lucene.apache.org
> Subject: Re: MergeFactor advice wanted
>
> I think it is worth your time to do some benchmarking. I think
> mergeFactor is not very helpful in the end...if you set it high, you'll
> index faster but then your searches will be slower prompting you to
> optimize...after which you'll find that you paid all your gains back.
> Test things out for yourself, but I'd recommend a low merge factor and
> then you can forget about the hassle of optimizing. Amortize, amortize...
>
> - Mark
>
> Aleksander M. Stensby wrote:
> > Hello everyone!
> > I'm wondering if any of you have any helpful advice to what
> > MergeFactor i should use...
> > The indexing process is handling a large amount of documents and i
> > would like to index as fast as possible.
> > Initial thought was to increase the mergeFactor to make the indexer
> > work more in memory and less writing to file. Thus this created a
> > problem for me with "TOO-MANY-OPEN-FILES"... of course, since i choose
> > 2000 as my mergeFactor:) Well, i could do an optimize from time to
> > time, but the big question is whats more efficient? Optimize tends to
> > take a loooong time on our system since it is quite a large index.
> >
> > Any helpful advice to what i should do? 10 as mergeFactor cant
> > possibly be the best solution here? Any advice would be highly
> > appreciated!
> >
> > - Aleksander
> >
> > --Aleksander M. Stensby
> > Software Developer
> > Integrasco A/S
> > aleksander.stensby@integrasco.no
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message