Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of markrmiller@gmail.com
 designates 64.233.166.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding;
        b=PPWuef7pBV5Cacp+IfIA7WJ7meY7RiEbbfeQqiX311/OtxmVvk0DGsvDpMCMLw3cz44BJhdJB4wsHBweEHS+c0NjYipVxLpR1cNUySR2krsKhwMIH7UKi9KxP6/4IeGonO7VR4E8/wLqyI2/Yhe4vkSVjdD4KJRh852He7A1k90=
Message-ID: <47A87A50.1080709@gmail.com>
Date: Tue, 05 Feb 2008 10:01:36 -0500
From: Mark Miller <markrmiller@gmail.com>
User-Agent: Thunderbird 2.0.0.9 (Windows/20071031)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Concurrent Indexing + Searching
References: <15234463.post@talk.nabble.com> <15255394.post@talk.nabble.com>
 <47A60308.4040300@gmail.com> <15262305.post@talk.nabble.com>
 <47A70844.8030806@gmail.com> <15288452.post@talk.nabble.com>
 <47A85527.6060207@gmail.com> <15289328.post@talk.nabble.com>
In-Reply-To: <15289328.post@talk.nabble.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit


>  Again, if the
> indexerThreads are bombarding the writer continuously, then the moment, when
> no indexer is accessing the writer, may never come. Thus, I invested some of
> my time, and wrote my own code, to control the sleeping of indexerThreads.
>   
I don't know how much of a concern this is. All you can really do is 
juggle the capabilities of Lucene, and Lucene was not designed to allow 
continuous writes to the database that are instantly available. That is 
one of the compromises of doing full text search over db. If you reopen 
the index in the face of constant write bombardment, it will already 
need to be reopened again immediately, and so on. You still need to 
consider the cost of reopening huge indexes...its not going to be fast 
enough to keep up with this kind of bombardment. I think you have to 
limit the use case.

I suppose you could refresh the readers occasionally in a long line of 
Writer get/release bombardment, but Lucene is just not in a position to 
handle such an interactive index, and I don't think it will be too 
fruitful trying to force it. If you correctly batch load, this is not 
that big of a limitation. Updates generally come in two ways...random 
updates here and there or a batch of updates at once - neither of these 
cases will cause bombardment.

- Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org