lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From heritrix.lucene <heritrix.luc...@gmail.com>
Subject Re: addIndexes() is taking infinite time ...
Date Thu, 22 Jun 2006 03:43:28 GMT
hi Otis,
Now this time it took 10 Hr 34 Min. to merge the indexes. During merging i
noticed it was not completey using the CPU. I have 512MB RAM. and here i
found it used upto the 256 MB.

Are there some more possibilities to make it more fast ...

With Regards,



On 6/21/06, heritrix. lucene <heritrix.lucene@gmail.com> wrote:
>
> hi,
> thanks for your reply.
> Now i restarted my application with maxBufferedDocs=10,000.
> And i am sorry to say that i was adding those indexes one by one. :-)
>
> Anyway Can you please explain me the addIndex ? I want to know what
> exactly happens while adding these..
>
> With Regards,
>
>
>
> On 6/20/06, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> >
> > If you can tell how many indices you've merged, you must be mering them
> > one at a time, and the pre and post merge optimize() calls are costing you.
> >
> > Also, that maxBufferedDocs looks pretty low.  Unless you are working
> > with very large documents and small heap, you should be able to bump that up
> > much higher.  I've used 10,000+ in some cases.
> >
> > Otis
> >
> > ----- Original Message ----
> > From: heritrix.lucene <heritrix.lucene@gmail.com>
> > To: java-user@lucene.apache.org
> > Sent: Tuesday, June 20, 2006 8:07:19 AM
> > Subject: addIndexes() is taking infinite time ...
> >
> > Hi all,
> > I had five different indexes:
> > 1 having 15469008 documents
> > 2 having 7734504 documents
> > 3 having 7734504 documents
> > 4 having 7734504 documents
> > 5 having 7734504 documents
> > Which sums to 46407024.
> > The constant values are
> > maxMergeFactor = 1000
> > maxBufferedDocs = 1000
> >
> > I wrote a simple program which uses the addIndex method for adding
> > indexes.
> > It has been more then 32 hours adding the indexes. My logs say upto now
> > it
> > has finished only first two indexes. It is adding the third one.
> > I want to know what exactly happens while merging the indexes?? Why this
> > time grows exponentially ????
> > Can anybody explain this in brief.
> >
> > Thanks in advance..
> > With Regards
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message