Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of yseeley@gmail.com designates
 64.233.166.179 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=oM2C0t4U8bTRNoJbuLtZnoixzJc+27zQa3dhFN6qPfkMqpX3bbO8uR1Z0u8O0IxJUYw0HRw+dGH6Mv7sxu0J3ssI23YsHV1xKn2yGzm6VwlXN7ZQe7qwfq1cqfIOyQwsDP1bGbEG/tbYdW57nAVJ4GkinrwewXQalYf5We3Vpic=
Message-ID: <c68e39170606020930u34cb2800g79a3c1f5775dddb8@mail.gmail.com>
Date: Fri, 2 Jun 2006 12:30:52 -0400
From: "Yonik Seeley" <yseeley@gmail.com>
To: java-dev@lucene.apache.org, simon.willnauer@gmail.com
Subject: Re: GData Server - Lucene storage
In-Reply-To: <f18c9dde0606020917k3ca27a32y5bdee6482d166662@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <f18c9dde0606011637td73e20au3e19dc3970e390cb@mail.gmail.com>
	 <c68e39170606020627m4a4e3e72yd603e6a091ee5995@mail.gmail.com>
	 <f18c9dde0606020643q30a09c55iedaa0b76b472eb20@mail.gmail.com>
	 <c68e39170606020855qae9e106k4da913a046e0f99e@mail.gmail.com>
	 <f18c9dde0606020917k3ca27a32y5bdee6482d166662@mail.gmail.com>

On 6/2/06, Simon Willnauer <simon.willnauer@googlemail.com> wrote:
> The problem here is that there is no action comparable to commit. The entry
> comes in and will be added to the storage. The delete will be queued but
> when should the delete operation start. Waiting for the writer to idle?! We
> could do it that way. but if a search request comes in the old entries will
> be found an can be retrieved from the storage. In that case I have to hold
> the already added but not yet deleted entries in a storage cache to prefent
> the storage from retrieving outdated and updated entries because they have
> the same ID.
>
> You use multiple indexSearcher instances to serve searches right?

There is normally only a single searcher open at a time (except when warming).
A main "registered" searcher handles all live requests.  When a new
searcher is opened, it is warmed up (requests run against it in the
background, caches pre-populated, etc) and then it is registered.

> so when
> all the deletes are done you have to reopen all indexsearchers again right?!

Just one.

> So this would happen quiet often due to updates and inserts.
> Hmm it is more and more a bad idea to use a lucene index as a storage.
> Rather go straight to a Database.

Yes, if you need to be able to *instantly* retrieve (but not search)
updates you just inserted, and you need to support a high volume of
updates and queries.

You could also do that in-memory by supporting retrieval by id from
your "batch" of documents not yet "committed" in Lucene.  Only
downside is it's volatile.

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org