Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of erickerickson@gmail.com
 designates 209.85.221.127 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=pbN96PVY7u14K5RZ0+8Vxjb2FT7LqtwpXerww6jBVQXQ9c2meU9VVF2zenN+Rg6KzU
         PnGCCMmi7yoAWsCZx/P3sAOkf2jHBGdFjySv6gHyhzCVYOlXvrw54sv1laTEvTnSP2sx
         lhhqK26hVG/TOaR3+iGg94Iu40dlsvPAuQylA=
MIME-Version: 1.0
In-Reply-To: <d36c28c30904300323pc0c551fm354e9e4a6b6fb3c2@mail.gmail.com>
References: <d36c28c30904300029t6414e3fapf4a36619baeabed9@mail.gmail.com>
	 <8c4e68610904300130m44a7e41ekb830af82bbe5f8dd@mail.gmail.com>
	 <d36c28c30904300323pc0c551fm354e9e4a6b6fb3c2@mail.gmail.com>
Date: Thu, 30 Apr 2009 09:46:21 -0400
Message-ID: <359a92830904300646v3120eed8if0768103a707228b@mail.gmail.com>
Subject: Re: Indexing becomes slow with time
From: Erick Erickson <erickerickson@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=0016e6470526de18550468c5eb24

--0016e6470526de18550468c5eb24
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

This is surprising behavior, which is another way of saying that,
given what you've said so far, this shouldn't be happening. I'd
really look at system metrics, like whether you're swapping
etc. In particular you might want to try varying how big you
allow your memory footprint to grow before you flush, this is
in the doc Ian pointed out under *
Flush by RAM usage instead of document count*

There's no need to periodically optimize, just do that at the end
if you must.

Best
Erick

On Thu, Apr 30, 2009 at 6:23 AM, liat oren <oren.liat@gmail.com> wrote:

> Yes, I do run optimize...
>
> I did start looking at these tips in the last few days, but didn't think
> the
> optimize makes it so slow.
>
> Thanks!
>
> 2009/4/30 Ian Lea <ian.lea@gmail.com>
>
> > Are you maybe running optimize after every n documents?  There are
> > lots of tips in
> > http://wiki.apache.org/lucene-java/ImproveIndexingSpeed.
> >
> >
> > --
> > Ian.
> >
> >
> > On Thu, Apr 30, 2009 at 8:29 AM, liat oren <oren.liat@gmail.com> wrote:
> > > Hi,
> > >
> > > I noticed that when I start to index, it indexes 7 documents a second.
> > After
> > > 30 minutes it goes down to 3 documents a second.
> > > After two hours it becomes very slow (I stopped it when it arrived to
> > 320MB
> > > and did 1 document in almost a minute)
> > >
> > > As you can see, it happens only after 2000, 3000 documnet.
> > > Should I split them into more indexes?
> > >
> > >
> > > Thanks,
> > > Liat
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>

--0016e6470526de18550468c5eb24--