lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David K <dkas...@asite.com>
Subject Re: Multiple instances of Lucene IndexWriter
Date Fri, 12 Oct 2007 15:32:42 GMT

I can't really say I'm "very familiar with the Lucene internals" :-)

What method would you recommend for checking for locked indexes? I have seen
mainly two methods and would be interested in the faster one with less
overhead:

Directory directory = FSDirectory.getDirectory(indexDir);
directory.makeLock(IndexWriter.WRITE_LOCK_NAME).isLocked()

or
Directory directory = FSDirectory.getDirectory(indexDir);
IndexReader.isLocked(directory)

many thanks,
David


Fredrik Andersson-2 wrote:
> 
> What you suggested is generally the most easygoing way to deal with
> it, i.ehaving a separate index per writer and one serial merging
> process. I have
> dabbled with disabling (file system) locks and synchronizing the writing
> processes by different means, but it's failure-prone unless you're very
> familiar with the Lucene internals.
> So, if it isn't a big hassle to create a serial merger (depends on your
> hardware/communiction setup mostly I guess) I would recommend that.
> 
> On 10/12/07, David K <dkaspar@asite.com> wrote:
>>
>>
>> Thank you for the quick response but at the moment we are interested in
>> our
>> own (small) usage of Lucene. It may be that in the future it turns out
>> that
>> Solr is the solution we need.
>>
>> At the moment I was hoping for a more descriptive workaround for the
>> issue
>> of using multiple instances of IndexWriter on the same index.
>>
>>
>>
>>
>> Erik Hatcher wrote:
>> >
>> > David,
>> >
>> > Have a look at Solr, http://lucene.apache.org/solr - it addresses
>> > this issue and many others that you would likely encounter with using
>> > pure Lucene.
>> >
>> >       Erik
>> >
>> >
>> > On Oct 12, 2007, at 6:26 AM, David K wrote:
>> >
>> >>
>> >> We are currently evaluating Lucene for document indexing and a
>> >> question came
>> >> up regarding multiple instances of IndexWriter possibly accessing
>> >> the same
>> >> index (directory).
>> >>
>> >> This would be a consequence of multiple instances of our application
>> >> possibly accessing the same index. Where multiple instances are
>> >> used for
>> >> load balancing and fail over of the application.
>> >>
>> >> The index could be either on a local drive when virtualization is
>> >> used to
>> >> achieve multiple instances on a single box. The index could also be
>> >> on a
>> >> shared drive (windows file sharing) and multiple server instances
>> >> trying to
>> >> update it.
>> >>
>> >> I have been looking around in the forums and it is always advised
>> >> against
>> >> multiple instances of IndexWriter  writing to the same index but I was
>> >> wondering whether the group has any suggestions for workarounds.
>> >> Surely
>> >> there must be other load balanced applications using Lucene?
>> >>
>> >> Some the workarounds I can think of OTTOMH:
>> >>
>> >> 1. each instance writing to a local index and merge these local
>> >> indexes
>> >> periodically to a shared index where searching is performed
>> >>
>> >> 2. implement our own queuing algorithms by testing for write locks
>> >> and wait
>> >> until locks are cleared
>> >>
>> >> thank you,
>> >> David
>> >> --
>> >> View this message in context: http://www.nabble.com/Multiple-
>> >> instances-of-Lucene-IndexWriter-tf4612568.html#a13172543
>> >> Sent from the Lucene - General mailing list archive at Nabble.com.
>> >
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Multiple-instances-of-Lucene-IndexWriter-tf4612568.html#a13174201
>> Sent from the Lucene - General mailing list archive at Nabble.com.
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/Multiple-instances-of-Lucene-IndexWriter-tf4612568.html#a13177008
Sent from the Lucene - General mailing list archive at Nabble.com.


Mime
View raw message