lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: [jira] Commented: (LUCENE-1026) Provide a simple way to concurrently access a Lucene index from multiple threads
Date Thu, 29 Nov 2007 15:59:28 GMT
Thanks Robert. Ill keep the sync then. I only considered it possible as 
the read is for reporting type purposes and so is not relied on for 
functionality. Sounds like we better retain the sync anyway though.


I have incorporated your code into mine. Looks great so far.

There is a rather large MultiIndexAccessor issue that I have to deal with though. It may turn
out that MultiSearchers may not be able to be cached. Without passing information about Writer
releases to the MultiIndexAccessor, the MultiIndexAccessor will hold onto old Searchers that
it shouldn't (the cache never gets cleared). The simple solution is to take out the MultiSearcher
caching and just make a new one on every request. This is not a huge deal as the underlying
Searchers will be cached, but I liked avoiding the calls to getIndexAcessor if possible. Its
prob best to drop the cache though. I think if this is done, your close method on MultiIndexAccessor
can  be removed.

I have added to the tests to expose this issue.

I will post updated code later.

- Mark

robert engels wrote:
> FYI, any time one thread reads a value while another thread updates it 
> it needs to be synchronized, or with current JVMs a volatile variable.
> The Java Memory Model requires this.
> Otherwise you can get a partial value (when the underlying value 
> requires more than one memory access to retrieve).
> On Nov 29, 2007, at 6:43 AM, Mark Miller (JIRA) wrote:
>>     [ 

>> ]
>> Mark Miller commented on LUCENE-1026:
>> -------------------------------------
>> Hey Mark, few more questions:
>> 1. Why is StopWatch needed?
>> StopWatch could certainly be taken out. I simply used it for 
>> show someone else that it doesn't take that long to get 
>> an accessor. I have no problem with removing it.
>> 2. No need to synchronize on this since you're only returning the 
>> value, not performing any modifications to it.
>> Its possible this could be taken out. I doubt there would be any 
>> noticeable trouble if you tried to read that value while another 
>> thread was updating it.
>> 3. open() - is there a real need to throw an exception if someone 
>> called the twice?
>> No real need I suppose. I believe this came in from the original 
>> code. I have no objections to taking it out.
>> SimpleSearchServer is an attempt to give an example that shows how 
>> this code can be used to create a very simple class that allows 
>> multi-threaded access to a Lucene index. I whipped it out extremely 
>> quickly, but its something I would like to build on. It can be nice 
>> to hide the IndexAccessor code behind a simple to use class that 
>> hides Lucene and a lot of complexity.
>> Thanks for your work on this!
>> - Mark
>>> Provide a simple way to concurrently access a Lucene index from 
>>> multiple threads
>>> --------------------------------------------------------------------------------

>>>                 Key: LUCENE-1026
>>>                 URL:
>>>             Project: Lucene - Java
>>>          Issue Type: New Feature
>>>          Components: Index, Search
>>>            Reporter: Mark Miller
>>>            Priority: Minor
>>>         Attachments:, 
>>> For building interactive indexes accessed through a network/internet 
>>> (multiple threads).
>>> This builds upon the LuceneIndexAccessor patch. That patch was not 
>>> very newbie friendly and did not properly handle MultiSearchers (or 
>>> at the least made it easy to get into trouble).
>>> This patch simplifies things and provides out of the box support for 
>>> sharing the IndexAccessors across threads. There is also a simple 
>>> test class and example SearchServer to get you started.
>>> Future revisions will be zipped.
>>> Works pretty solid as is, but could use the ability to warm new 
>>> Searchers.
>> -- 
>> This message is automatically generated by JIRA.
>> -
>> You can reply to this email to add a comment to the issue online.
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message