Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: domain of erickerickson@gmail.com
 designates 209.85.134.191 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=nbiTMov8hnJHKCelGfM6HkahVQj4BcJKPYFY/C9UpQKWdjqr53UQkAhYYISVFbqlZ6iQmetZPAFJfdi+aY/TEEvL0JniWrbUSOyb2dT6K5CIDdNW+65lJSRD9RoMKoWKjb7FmJhZF3Q/WOGur0Dn7VvJoh5AsovmAaJ0SU+zvfo=
Message-ID: <359a92830705240628r5902dd41rcb5c7bceade2014b@mail.gmail.com>
Date: Thu, 24 May 2007 09:28:30 -0400
From: "Erick Erickson" <erickerickson@gmail.com>
To: java-user@lucene.apache.org
Subject: Re: Searching on a Rapidly changing Index
In-Reply-To: <20070524092205.GH5135@thegestalt.org>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_83970_26999198.1180013310910"
References: <20070524092205.GH5135@thegestalt.org>

------=_Part_83970_26999198.1180013310910
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Another option would be to only re-open your searcher when actually
needed, that is after the index has changed. This only does you some
good when you have some hope that there are sizable gaps in
your modifications....

Another possibility is to relax the "immediately" constraint. Would
a maximum of 10 minutes really be unacceptable? That would
allow you to open up a new Reader every 10 minutes, warm it up
in the background and then switch.

This works if you can say something to your product manager like
"give me a 10 minute delay and people will get significantly faster
results".

If that's unacceptable, you can *still* open up a new reader in the
background and warm it up before using it. "immediately" then
becomes 5-10 seconds or so.

Best
Erick

On 5/24/07, Simon Wistow <simon@thegestalt.org> wrote:
>
> I've built a Lucene system that gets rapidly updated - documents are
> supposed to be searchable immeidately after they've been indexed.
>
> As such I have a Writer that puts new index, update and delete tasks
> into a queue and then has a thread which consumes them and applies them
> to the index using one instance of an IndexModifier. So far so good.
>
> However every new Search opens up a new IndexSearcher in order to make
> sure that it gets new documents. The docs indicate is a big
> performance penalty.
>
> Are there any tips or tricks I can use to mitigate this?
>
> thanks,
>
> Simon
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

------=_Part_83970_26999198.1180013310910--