lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Parallel Index Search
Date Mon, 16 Oct 2006 13:41:12 GMT
Supriya Kumar Shyamal wrote:

> If I am not mistaken the process of locking the Index by different 
> objects like IndexReader or Indexwriter, theoratically only one Thread 
> can access the index at a time.

Actually, only one writer can write to the index at once.  Multiple
readers can read from the index.

On top of that, multiple threads may share one writer and one reader.
Ie, these classes are thread safe.

> When we do search on the index it creates a commit lock so the other 
> thread does not modify the index, so other thread waits until the 
> previosu therads release the lock, is it right?
> 
> So in this case I should say index accessed one by one not parallel?

The commit lock is only held while a reader is loading the index and
while a writer is "committing" its changes to the index.  These times
should be brief.  Whereas, the write lock is held for the entire time
that a writer is open.

> Its just my speculation, please don't get me wrong.
> 
> Because I try to share the same index by 6 instances and since the lock 
> for 5 instances are disabled and only once instance can modify the 
> index, at this case I achive the parallel read of the index. Only 
> disadvantage is that when the index modified then I get 
> FileNotFoundException, ao I do some kind of  respawn the search again.

You should not have to disable locking to do this sharing; in fact,
disabling locking will lead to the "FileNotFoundException" on
instantiating a reader when a writer is committing.

> If I implement the lock mechanism in the DB using the custom locking 
> then I am afraid the index performance will be reduced but the only 
> advantage is that I can avoid FNFE.

There are also open issues at least over NFS eg:

     http://issues.apache.org/jira/browse/LUCENE-673

Whereby even with proper (native) locking it's still possible to
hit exceptions due to caching in the NFS client.  Are you using NFS?

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message