lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller" <markrmil...@gmail.com>
Subject Re: thread safe shared IndexSearcher
Date Wed, 19 Sep 2007 19:44:18 GMT
Ill respond a point at a time:

1.

****************************** Hi Maik,

So what happens in this case:

IndexAccessProvider accessProvider = new IndexAccessProvider(directory,

analyzer);

LuceneIndexAccessor accessor = new LuceneIndexAccessor(accessProvider);

accessor.open();

IndexWriter writer = accessor.getWriter();

// reference to the same instance?

IndexWriter writer2 = accessor.getWriter();

writer.addDocument(....);

writer2.addDocument(....);



// I didn't release the writer yet

// will this block?

IndexReader reader = accessor.getReader();

reader.delete(....);

************

This is not really an issue. First, if you are going to delete with a Reader
you need to call getWritingReader and not getReader. When you do that, the
getWritingReader call will block until writer and writer2 are released. If
you are just adding a couple docs before releasing the writers, this is no
problem because the block will be very short. If you are loading tons of
docs and you want to be able to delete with a Reader in a timely manner, you
should release the writers every now and then (release and re-get the Writer
every 100 docs or something). An interactive index should not hog the
Writer, while something that is just loading a lot could hog the Writer.
This is no different than normal…you cannot delete with a Reader while
adding with a Writer with Lucene. This code just enforces those semantics.
The best solution is to just use a Writer to delete – I never get a
ReadingWriter.

2. http://issues.apache.org/bugzilla/show_bug.cgi?id=34995#c3

This is no big deal either. I just added another getWriter call that takes a
create Boolean.

3. I don't think there is a latest release. This has never gotten much
official attention and is not in the sandbox. I worked straight from the
originally submitted code.

4. I will look into getting together some code that I can share. The
multisearcher changes that are need are a couple of one liners really, so at
a minimum I will give you the changes needed.



-       Mark



On 9/19/07, Jay Yu <yu@ai.sri.com> wrote:

Mark,



thanks for sharing your insight and experience about LuceneIndexAccessor!

I remember seeing some people reporting some issues about it, such as:

http://www.archivum.info/java-dev@lucene.apache.org/2005-05/msg00114.html

http://issues.apache.org/bugzilla/show_bug.cgi?id=34995#c3



Have those issues been resolved?



Where did you get the latest release? It is not in the official Lucene

sandbox/contrib.



Finally, are you willing to share your extended version to include your

tweak relating to the MultiSearcher?



Thanks a lot!



Jay



Mark Miller wrote:

> I use option 3 extensivley and find it very effective. There is a tweak or

> two required to get it to work right with MultiSearchers, but other than

> that, the code is great. I have built a lot on top of it. I'm on the list

> all the time and would be happy to answer any questions you have in
regards

> to LuceneIndexAccessor. Frankly, I think its overlooked far too much.

>

> - Mark

>

>

> On 9/19/07, Jay Yu <yu@ai.sri.com> wrote:

>>

>> In a multithread app like web app, a shared IndexSearcher could throw a

>> AlreadyClosedException when another thread is trying to update the

>> underlying IndexReader by closing the shared searcher after the index is

>> updated. Searching over the past discussions on this mailing list, I

>> found several approaches to solve the problem.

>> 1. use solr

>> 2. use DelayCloseIndexSearcher

>> 3. use LuceneIndexAccessor

>>

>>

>> the first one is not feasible for us; some people seemed to have

>> problems with No. 2 and I do not find a lot of discussions around No.3.

>>

>> I wonder if anyone has good experience on No 2 and 3?

>> Or do I miss other better solutions?

>>

>> Thanks for any suggestion/comment!

>>

>> Jay

>>

>> ---------------------------------------------------------------------

>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org

>> For additional commands, e-mail: java-user-help@lucene.apache.org

>>

>>

>



---------------------------------------------------------------------

To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org

For additional commands, e-mail: java-user-help@lucene.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message