lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Lucene write locks
Date Fri, 25 Jul 2008 12:38:29 GMT

It's not enough to have a synchronized block only, unless eg within  
the synchronized block you open the IndexWriter, add docs, close it.

Basically, you must also ensure only one instance of IndexWriter is  
open at once.

If you try to open another instance of IndexWriter while a previous  
instance is still open, you'll get that LockObtainFailedException.

Alternatively, on hitting that exception you could pause & retry a bit  
later if you want.  But that solution is non-optimal because you are  
inserting unnecessary delay in when a document gets indexed.

Mike

Sandeep K wrote:

>
> Hi Mike,
> sorry i didn't get much of it.
> i tried with synchronized block also.
> but it was a failure. will it be enough to have
> a synchronized block only?
>
> if each instance of IndexWriter is thread safe,
> then do we need to check whether its open or closed each time?
> why it's not doing the work even if an IndexWriter instance is
> created for each file to be indexed?
>
> thanks and regards,
> sand
>
>
>
> Michael McCandless-2 wrote:
>>
>>
>> I think the simplest approach is to use a synchronized block in your
>> application to check if an IndexWriter is currently open, and if so,
>> use that one to add your docs, else, open a new one and store it in a
>> central place where the next message to come in can go and find it?
>>
>> Each instance of IndexWriter is thread safe, so it's fine if multiple
>> threads add documents at the same time.
>>
>> Mike
>>
>> Sandeep K wrote:
>>
>>>
>>> Hey Michael,
>>> Thanks a lot for ur time.
>>> On the JMS side I have the code which uses IndexWriter.
>>> I have to index the files that are getting uploaded. It's fine
>>> all the way if i upload ane file. But when i tried with 2-3 files,
>>> it giving me the following error.
>>>
>>> org.apache.lucene.store.LockObtainFailedException:
>>> Lock obtain timed out: [*Lucene_store_path]/write.lock
>>>
>>> I guess this is because while indexing the second file, it tries to
>>> get the
>>> lock
>>> before the completion of the first one. I am planning to use
>>> Synchronized
>>> blocks.
>>> Is there any other way to stop this or make it synchronized?
>>> ie. to make it sure that only one indexing happens at a time?
>>>
>>> thanks a lot
>>> regards,
>>> sand
>>>
>>>
>>> Michael McCandless-2 wrote:
>>>>
>>>>
>>>> OK then this should be fine.  That single machine, on receiving a  
>>>> JMS
>>>> message, should use a single IndexWriter for making changes to the
>>>> index (ie, it should not try to open a 2nd IndexWriter while a
>>>> previous one is still working on a previous message).
>>>>
>>>> Mike
>>>>
>>>> Sandeep K wrote:
>>>>
>>>>>
>>>>> Thanks a lot Mike,
>>>>> There will be only one machine which uses IndexWriter and its  the
>>>>> JMS server. This server will first create the file in the physical
>>>>> file
>>>>> system(its Linux)
>>>>> and then index the saved file.
>>>>>
>>>>>
>>>>> Michael McCandless-2 wrote:
>>>>>>
>>>>>>
>>>>>> Sandeep K wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi all..
>>>>>>> I had a question related to the write locks created by Lucene.
>>>>>>> I use Lucene 2.3.2. Will this newwer version create locks while
>>>>>>> indexing as
>>>>>>> older ones?
>>>>>>> or is there any other way that lucene handles its operations?
>>>>>>
>>>>>> It still creates write locks, which are used to ensure only one
>>>>>> writer
>>>>>> is changing the index at a time.
>>>>>>
>>>>>>> And my another doubt is that i use JMS for lucene indexing.
>>>>>>> My App server will not do indexing but will pass the needed data
>>>>>>> for
>>>>>>> indexing to the JMS server.
>>>>>>> will there be any problem in indexing as its asynchronous?
>>>>>>> plz help me..
>>>>>>
>>>>>> Will there be multiple machines that might open an IndexWriter
>>>>>> and do
>>>>>> indexing, on receiving messages from the JMS server?
>>>>>>
>>>>>> What filesystem is being used to share access to the index?
>>>>>>
>>>>>> Mike
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> -- 
>>>>> View this message in context:
>>>>> http://www.nabble.com/Lucene-write-locks-tp18604932p18607685.html
>>>>> Sent from the Lucene - Java Users mailing list archive at
>>>>> Nabble.com.
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>
>>> -- 
>>> View this message in context:
>>> http://www.nabble.com/Lucene-write-locks-tp18604932p18646366.html
>>> Sent from the Lucene - Java Users mailing list archive at  
>>> Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>
> -- 
> View this message in context: http://www.nabble.com/Lucene-write-locks-tp18604932p18649403.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message