lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Indexing becomes slow with time
Date Thu, 30 Apr 2009 13:46:21 GMT
This is surprising behavior, which is another way of saying that,
given what you've said so far, this shouldn't be happening. I'd
really look at system metrics, like whether you're swapping
etc. In particular you might want to try varying how big you
allow your memory footprint to grow before you flush, this is
in the doc Ian pointed out under *
Flush by RAM usage instead of document count*

There's no need to periodically optimize, just do that at the end
if you must.

Best
Erick

On Thu, Apr 30, 2009 at 6:23 AM, liat oren <oren.liat@gmail.com> wrote:

> Yes, I do run optimize...
>
> I did start looking at these tips in the last few days, but didn't think
> the
> optimize makes it so slow.
>
> Thanks!
>
> 2009/4/30 Ian Lea <ian.lea@gmail.com>
>
> > Are you maybe running optimize after every n documents?  There are
> > lots of tips in
> > http://wiki.apache.org/lucene-java/ImproveIndexingSpeed.
> >
> >
> > --
> > Ian.
> >
> >
> > On Thu, Apr 30, 2009 at 8:29 AM, liat oren <oren.liat@gmail.com> wrote:
> > > Hi,
> > >
> > > I noticed that when I start to index, it indexes 7 documents a second.
> > After
> > > 30 minutes it goes down to 3 documents a second.
> > > After two hours it becomes very slow (I stopped it when it arrived to
> > 320MB
> > > and did 1 document in almost a minute)
> > >
> > > As you can see, it happens only after 2000, 3000 documnet.
> > > Should I split them into more indexes?
> > >
> > >
> > > Thanks,
> > > Liat
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message