lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Yu ...@AI.SRI.COM>
Subject Re: thread safe shared IndexSearcher
Date Wed, 19 Sep 2007 20:22:08 GMT
Thanks for your detailed explanation of the issues and your solutions.
It seems that LuceneIndexAccessor is worth trying first before I 
implement other locking mechanism to ensure proper order.
I will appreciate it very much if you'd like your extension with us.

Jay

Mark Miller wrote:
> Ill respond a point at a time:
> 
> 1.
> 
> ****************************** Hi Maik,
> 
> So what happens in this case:
> 
> IndexAccessProvider accessProvider = new IndexAccessProvider(directory,
> 
> analyzer);
> 
> LuceneIndexAccessor accessor = new LuceneIndexAccessor(accessProvider);
> 
> accessor.open();
> 
> IndexWriter writer = accessor.getWriter();
> 
> // reference to the same instance?
> 
> IndexWriter writer2 = accessor.getWriter();
> 
> writer.addDocument(....);
> 
> writer2.addDocument(....);
> 
> 
> 
> // I didn't release the writer yet
> 
> // will this block?
> 
> IndexReader reader = accessor.getReader();
> 
> reader.delete(....);
> 
> ************
> 
> This is not really an issue. First, if you are going to delete with a Reader
> you need to call getWritingReader and not getReader. When you do that, the
> getWritingReader call will block until writer and writer2 are released. If
> you are just adding a couple docs before releasing the writers, this is no
> problem because the block will be very short. If you are loading tons of
> docs and you want to be able to delete with a Reader in a timely manner, you
> should release the writers every now and then (release and re-get the Writer
> every 100 docs or something). An interactive index should not hog the
> Writer, while something that is just loading a lot could hog the Writer.
> This is no different than normal…you cannot delete with a Reader while
> adding with a Writer with Lucene. This code just enforces those semantics.
> The best solution is to just use a Writer to delete – I never get a
> ReadingWriter.
> 
> 2. http://issues.apache.org/bugzilla/show_bug.cgi?id=34995#c3
> 
> This is no big deal either. I just added another getWriter call that takes a
> create Boolean.
> 
> 3. I don't think there is a latest release. This has never gotten much
> official attention and is not in the sandbox. I worked straight from the
> originally submitted code.
> 
> 4. I will look into getting together some code that I can share. The
> multisearcher changes that are need are a couple of one liners really, so at
> a minimum I will give you the changes needed.
> 
> 
> 
> -       Mark
> 
> 
> 
> On 9/19/07, Jay Yu <yu@ai.sri.com> wrote:
> 
> Mark,
> 
> 
> 
> thanks for sharing your insight and experience about LuceneIndexAccessor!
> 
> I remember seeing some people reporting some issues about it, such as:
> 
> http://www.archivum.info/java-dev@lucene.apache.org/2005-05/msg00114.html
> 
> http://issues.apache.org/bugzilla/show_bug.cgi?id=34995#c3
> 
> 
> 
> Have those issues been resolved?
> 
> 
> 
> Where did you get the latest release? It is not in the official Lucene
> 
> sandbox/contrib.
> 
> 
> 
> Finally, are you willing to share your extended version to include your
> 
> tweak relating to the MultiSearcher?
> 
> 
> 
> Thanks a lot!
> 
> 
> 
> Jay
> 
> 
> 
> Mark Miller wrote:
> 
>> I use option 3 extensivley and find it very effective. There is a tweak or
> 
>> two required to get it to work right with MultiSearchers, but other than
> 
>> that, the code is great. I have built a lot on top of it. I'm on the list
> 
>> all the time and would be happy to answer any questions you have in
> regards
> 
>> to LuceneIndexAccessor. Frankly, I think its overlooked far too much.
> 
> 
>> - Mark
> 
> 
> 
>> On 9/19/07, Jay Yu <yu@ai.sri.com> wrote:
> 
> 
>>> In a multithread app like web app, a shared IndexSearcher could throw a
> 
>>> AlreadyClosedException when another thread is trying to update the
> 
>>> underlying IndexReader by closing the shared searcher after the index is
> 
>>> updated. Searching over the past discussions on this mailing list, I
> 
>>> found several approaches to solve the problem.
> 
>>> 1. use solr
> 
>>> 2. use DelayCloseIndexSearcher
> 
>>> 3. use LuceneIndexAccessor
> 
> 
> 
>>> the first one is not feasible for us; some people seemed to have
> 
>>> problems with No. 2 and I do not find a lot of discussions around No.3.
> 
> 
>>> I wonder if anyone has good experience on No 2 and 3?
> 
>>> Or do I miss other better solutions?
> 
> 
>>> Thanks for any suggestion/comment!
> 
> 
>>> Jay
> 
> 
>>> ---------------------------------------------------------------------
> 
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> 
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 
> 
> 
> 
> ---------------------------------------------------------------------
> 
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> 
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message